---
title: "z - Advanced topic: Handling packages with remote dependencies"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{z-advanced-topic-handling-packages-with-remote-dependencies}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

```{r, include=FALSE}
library(rix)
```

## Introduction

Packages published on CRAN must have their dependencies on either CRAN or
Bioconductor, but not on GitHub. However, there are many packages available on
GitHub that never get published on CRAN, and some of these packages may even
depend on other packages that are also only available on GitHub. `{rix}` makes
it possible to install packages from GitHub and if these packages have
dependencies that are also on GitHub, these also get correctly added to the
generated `default.nix`.

There are however certain caveats you should be aware of.

## The {lookup} package

As an example we are going to use the
[{lookup}](https://github.com/jimhester/lookup) package which has only been
released on GitHub. [Here is the
repository](https://github.com/jimhester/lookup). This package comes with the
`lookup()` function which makes it possible to check the source code of any
function from a loaded package, even if the source of that function is in C or
Fortran. To create a reproducible development environment that makes
[{lookup}](https://github.com/jimhester/lookup) available to you, you could use
the following `rix::rix()` call:

```{r, eval = F}
path_default_nix <- tempdir()

rix(
  r_ver = "latest-upstream",
  r_pkgs = NULL,
  system_pkgs = NULL,
  git_pkgs = list(
    package_name = "lookup",
    repo_url = "https://github.com/jimhester/lookup/",
    commit = "eba63db477dd2f20153b75e2949eb333a36cccfc"
  ),
  ide = "other",
  project_path = path_default_nix,
  overwrite = TRUE,
  print = TRUE
)
```

This will generate the following `default.nix`:

```
let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/6a25f33c843a45b8d67ba782b6782973a7265774.tar.gz") {};

    httr2 = (pkgs.rPackages.buildRPackage {
      name = "httr2";
      src = pkgs.fetchgit {
        url = "https://github.com/r-lib/httr2";
        rev = "HEAD";
        sha256 = "sha256-UgJCFPO47mgUt3esRRPhXjr0oNDRrR9XAAIxMhZYbFc=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages)
          cli
          curl
          glue
          lifecycle
          magrittr
          openssl
          R6
          rappdirs
          rlang
          vctrs
          withr;
      };
    });

    gh = (pkgs.rPackages.buildRPackage {
      name = "gh";
      src = pkgs.fetchgit {
        url = "https://github.com/gaborcsardi/gh";
        rev = "HEAD";
        sha256 = "sha256-VpxFIfUEk0PudytQ3boMhEJhT0AnelWkSU++WD/HAyo=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) 
          cli
          gitcreds
          glue
          ini
          jsonlite
          lifecycle
          rlang;
      } ++ [ httr2 ];
    });


    highlite = (pkgs.rPackages.buildRPackage {
      name = "highlite";
      src = pkgs.fetchgit {
        url = "https://github.com/jimhester/highlite";
        rev = "HEAD";
        sha256 = "sha256-lkWMlAi75MYxiBUYnLwxLK9ApXkWanA4Mt7g4qtLpxM=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) 
          Rcpp
          BH;
      };
    });


    memoise = (pkgs.rPackages.buildRPackage {
      name = "memoise";
      src = pkgs.fetchgit {
        url = "https://github.com/hadley/memoise";
        rev = "HEAD";
        sha256 = "sha256-FDMNgrgctzkN8dXKRoWsOKe3tXxmm8Cqdu/Sh6WKx/Q=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) 
          rlang
          cachem;
      };
    });

    lookup = (pkgs.rPackages.buildRPackage {
      name = "lookup";
      src = pkgs.fetchgit {
        url = "https://github.com/jimhester/lookup/";
        rev = "eba63db477dd2f20153b75e2949eb333a36cccfc";
        sha256 = "sha256-arl7LVxL8xGUW3LhuDCSUjcfswX0rdofL/7v8Klw8FM=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) 
          Rcpp
          codetools
          crayon
          rex
          jsonlite
          rstudioapi
          withr
          httr;
      } ++ [ highlite gh memoise ];
    });

  system_packages = builtins.attrValues {
    inherit (pkgs)
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [ lookup   system_packages   ];
 
}
```

as you can see, several other packages hosted on GitHub were added
automatically. This is because these were listed as remote dependencies in
`{lookup}`'s `DESCRIPTION` file:

```
Remotes:
    jimhester/highlite,
    gaborcsardi/gh,
    hadley/memoise
```

## Caveats

`{highlite}` is a dependency of [{lookup}](https://github.com/jimhester/lookup)
that is only available on GitHub. `{gh}` and `{memoise}` are also listed as
remote dependencies, however, they are also available on CRAN. What likely
happened here was that `{gh}` and `{memoise}` were not yet available on CRAN at
the time when `{lookup}` was written (which was more than 6 years ago as of
2025). Because they are listed as remote dependencies, they will also be built
from GitHub instead of CRAN. Here, it is up to you to decide if you want to keep
the GitHub version of these packages, or if you should instead include the
released CRAN version. Depending on what you want to do, going for the CRAN
release of the packages might be advisable. For example in this case, trying to
build this expression will not work.

This is because `{httr2}` is a package that needs to be compiled from source and
which needs some Nix-specific fixes applied to its source code for it to build
successfully. Installing the version provided by `nixpkgs`, which builds upon
the released CRAN version will succeed however. To do so, change the
`default.nix` manually to this (essentially remove the definition of `{httr2}`
and put it as a `propagatedBuildInput` to `{gh}`):

```
let
 pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/6a25f33c843a45b8d67ba782b6782973a7265774.tar.gz") {};

    gh = (pkgs.rPackages.buildRPackage {
      name = "gh";
      src = pkgs.fetchgit {
        url = "https://github.com/gaborcsardi/gh";
        rev = "HEAD";
        sha256 = "sha256-VpxFIfUEk0PudytQ3boMhEJhT0AnelWkSU++WD/HAyo=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages)
          cli
          gitcreds
          glue
          httr2 # <- httr2 is now declared here, so it's the CRAN version
          ini
          jsonlite
          lifecycle
          rlang;
      };
    });

    highlite = (pkgs.rPackages.buildRPackage {
      name = "highlite";
      src = pkgs.fetchgit {
        url = "https://github.com/jimhester/highlite";
        rev = "HEAD";
        sha256 = "sha256-lkWMlAi75MYxiBUYnLwxLK9ApXkWanA4Mt7g4qtLpxM=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) 
          Rcpp
          BH;
      };
    });

    memoise = (pkgs.rPackages.buildRPackage {
      name = "memoise";
      src = pkgs.fetchgit {
        url = "https://github.com/hadley/memoise";
        rev = "HEAD";
        sha256 = "sha256-FDMNgrgctzkN8dXKRoWsOKe3tXxmm8Cqdu/Sh6WKx/Q=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages)
          rlang
          cachem;
      };
    });

    lookup = (pkgs.rPackages.buildRPackage {
      name = "lookup";
      src = pkgs.fetchgit {
        url = "https://github.com/jimhester/lookup/";
        rev = "eba63db477dd2f20153b75e2949eb333a36cccfc";
        sha256 = "sha256-arl7LVxL8xGUW3LhuDCSUjcfswX0rdofL/7v8Klw8FM=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages)
          Rcpp
          codetools
          crayon
          rex
          jsonlite
          rstudioapi
          withr
          httr;
      } ++ [ highlite gh memoise ];
    });

  system_packages = builtins.attrValues {
    inherit (pkgs)
      glibcLocales
      nix
      R;
  };

in

pkgs.mkShell {
  LOCALE_ARCHIVE = if pkgs.system == "x86_64-linux" then "${pkgs.glibcLocales}/lib/locale/locale-archive" else "";
  LANG = "en_US.UTF-8";
   LC_ALL = "en_US.UTF-8";
   LC_TIME = "en_US.UTF-8";
   LC_MONETARY = "en_US.UTF-8";
   LC_PAPER = "en_US.UTF-8";
   LC_MEASUREMENT = "en_US.UTF-8";

  buildInputs = [ lookup   system_packages   ];

}
```

In this manually edited expression, `{httr2}` will now build successfully
because Nix is instructed to build the CRAN version by applying [this
fix](https://github.com/NixOS/nixpkgs/blob/7b87fced8bc525d466c7258a042bd12ea336a3c6/pkgs/development/r-modules/default.nix#L1817)
which was added there by packagers and maintainers of the R programming language
for `nixpkgs` (it is exactly the same if you tried to install `{httr2}` from
GitHub on Windows: you would need to build it from source and thus make sure
that you have the required system-level dependencies to build it. Instead, it is
easier to install a pre-compiled binary from CRAN).

Another important point to address is that if remote dependencies are listed in
a `DESCRIPTION` file like this:

```
Remotes:
    jimhester/highlite,
    gaborcsardi/gh,
    hadley/memoise
```

`{rix}` will automatically use the latest commit from these repositories as the
revision. This also means that if these repositories are being actively worked on,
rebuilding these environments will actually pull another version of these packages.
Instead, it is advisable to edit the `default.nix` yet again, and replace mentions
of `HEAD` with an actual commit. For example, edit this:

```
    gh = (pkgs.rPackages.buildRPackage {
      name = "gh";
      src = pkgs.fetchgit {
        url = "https://github.com/gaborcsardi/gh";
        rev = "HEAD";
        sha256 = "sha256-VpxFIfUEk0PudytQ3boMhEJhT0AnelWkSU++WD/HAyo=";
      };
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages) 
          cli
          gitcreds
          glue
          ini
          jsonlite
          lifecycle
          rlang;
      } ++ [ httr2 ];
    });
```

to this:

```
    gh = (pkgs.rPackages.buildRPackage {
      name = "gh";
      src = pkgs.fetchgit {
        url = "https://github.com/gaborcsardi/gh";
        rev = "27db16cf363dc";
        sha256 = ""; # <- You will need to try to build the expression once, and then
      };             # <- put the sha256 that nix-build returns
      propagatedBuildInputs = builtins.attrValues {
        inherit (pkgs.rPackages)
          cli
          gitcreds
          glue
          ini
          jsonlite
          lifecycle
          rlang;
      } ++ [ httr2 ];
    });
```


However, if instead the remotes are listed like this:

```
Remotes:
    jimhester/highlite@abc123,
    gaborcsardi/gh@def123,
    hadley/memoise@ghi123
```

then the listed commits will be used, which will make sure that the build
process is reproducible. Only commits can be used, anything else listed there
(such as pull request numbers or release tags) will not work with `{rix}`.

In conclusion, `{rix}` makes it easier to build packages from GitHub which
have themselves dependencies hosted on GitHub, you should however make sure
that the expression that is generated uses fixed commits instead of `HEAD`
for the packages being built from GitHub, and you should also decide if you
want to use the version of a packages hosted on GitHub instead of the CRAN
release. These are decisions that `{rix}` cannot take for you.