Make it easier to integrate external derivations in nixpkgs

[José Luis Lafuente]

With more projects adding support for nix (especially with flakes gaining adoption), I see a duplication of efforts between nixpkgs maintainers and open source authors providing a nix derivation for their projects.

If an open source author includes a nix derivation in their project, to include that project in nixpkgs, someone has to manually keep in sync the nix files between the original project and nixpkgs.

If a project already provides a nix derivation, which only depends on nixpkgs, can't we reuse it for nixpkgs? That way, we could avoid that extra sync work.

There are two extra benefits if we can reuse upstream nix code in nixpkgs:

• Binary cache: While we can use a 3rd party binary cache (like cachix), for open source projects, it is more convenient to use the nixos cache (cache.nixos.org). That way, users don't have to worry about any extra configuration, like --accept-flake-config.
• Hydra CI: Open source authors have the benefit of getting tests executed by hydra

A possible solution could be to leverage passthru.updateScript. I'm exploring that approach, but I'd like to hear any suggestions or ideas about how to keep in sync nix derivations between nixpkgs and external open source projects.

[Silvan Mosberger]

PR: #218834

[José Luis Lafuente]

Thanks for your feedback @alyssais and @infinisil. I'm answering you to #218834 here, trying to keep the conversation in one place.

I created a new PR: #220530 exploring the option of committing the nix expression to Nixpkgs, and consume it on an upstream project. As an example, I created this project: https://github.com/jlesquembre/simple-flake (main branch)

The main idea is to add a json file (info.json in this example) and a new helper function srcFromJson. A derivation must accept 2 new arguments:

{
 srcFromJSON
, projectInfo ? srcFromJSON ./info.json
}:
stdenv.mkDerivation {
  pname = "simple";
  inherit (projectInfo) src version;
}

info.json looks like this:

{
  "fetcher": "fetchFromGitHub",
  "fetcherArgs": {
    "hash": "sha256-x9BrU5hj6HsnRudt7a7qEJOTX2WU1UdAxiPK40NLvew=",
    "owner": "jlesquembre",
    "repo": "simple-flake",
    "rev": "v1.0"
  },
  "version": "1.0"
}

srcFromJson function is calling the "fetcher" function with "fetcherArgs".

That way, it's easier to override the src value upstream, eg:

packages.default = pkgs.callPackage "${nixpkgs}/pkgs/applications/misc/simple-pkg/default.nix" {
  projectInfo = {
    src = ./.;
    version = "DEV";
  };
};

On top of that, a nice side effect of extracting the src data to a json file is that it makes easier to consume and manipulate that data by external programs.

I left the nixpkgsUpdater helper function, but with this new approach it's optional. It could be moved to a different PR.

[Alyssa Ross]

How is that any easier than using overrideAttrs?

[José Luis Lafuente]

Without the functionality to synchronize files from upstream to nixpkgs, indeed, there isn't much difference with overrideAttrs. I can't think of a better approach, but I still think the current situation is not ideal. IMO, there isn't a good way to share nix expressions between nixpkgs and upstream projects.