Local Nix Cache

This project tries to solve a common issue amongst people on low bandwidth internet connections: Sharing binaries between different NixOS systems without requiring an extended amount of trust to the local peers.

The primary idea is to re-serve the contents of the local Nix store in a format that is compatible with the signatures of cache.nixos.org. There is no additional trust setup required. As long you trust the signing key of hydra you can just use anyones NAR files.

Ideally we could just use IPFS but we aren't quiet there yet.

How do I use this?

Choose a machine to serve the cache from (called server from now on). It is useful to use a machine with a larger set of paths in the store or a machine that usually updates first.
Install the local-nix-cache daemon via the module from the module.nix (or your own) on the server.
On all the machines that should make a use of the paths that might already be present on the server add the server to nix.binaryCaches. e.g. nix.binaryCaches = [ "http://server:8083" "https://cache.nixos.org" ];
Run a build on a machine that isn't the server and you should see it downloading all the paths that are alreayd downloaded to the server from there. All the others will be retrieved from hydra as usual.

Use cases

Note: Not all of them a properly covered. Local discovery is pretty important to be really useful.

In general all situations where internet bandwidth or volume is a concern. E.g. slow links, high latencies, metered connections, ….

Some situations where this might come in handy are:

a bunch of machines with an overlapping set of packages downloading the same files.
a group of Nix(OS) users sharing an internet connection that falls in one of the previous categories. Conferences, Workshops, Co-worker in a shared office space.

(Offline scenarios are not (yet) supported. Hopefully some day.)

Requirements

At the present time it requires a patched Nix daemon to be running on the machine that serves it's nix store contents. The changes to vanilla Nix are very simple and expose another Operation via the store interface. The patch is available at https://github.com/andir/nix/tree/queryPathFromFileHash.

You can override the local Nix store in configuration.nix via:

{ pkgs, config, ...}: {
  nix.package = pkgs.nixUnstable.overrideAttrs (_: {
    src = pkgs.fetchFromGitHub {
      owner = "andir";
      repo = "nix";
      rev = "3aefaac46e4833b029555242f90d37f0df80f02f";
      sha256 = "0aq0k4kyjcml5dgbhgzf0iv2wl2psy650w80g1d33pnzzv6gcdsr";
    };
  });
}

How Can I use it?

{ config, pkgs, lib, ... }:
let
  local-nix-cache = import ((import <nixpkgs> {}).fetchFromGitHub {
    owner = "andir";
    repo = "local-nix-cache";
    rev = "be2d3ea9d8da586647540c164805730d2d34a556";
    sha256 = "1fdj2b7bz0gxdbbdc29x46ivlj8fkvg08w6jiya9xwqkr6gpv2ai";
  }) {};
in
{
  imports = [
    (local-nix-cache.path + "/module.nix")
  ];

  local-nix-cache = {
    server.enable = true;
    client.enable = true;
  };
  networking.firewall.allowedTCPPorts = [ config.local-nix-cache.server.port ];

  systemd.services.local-nix-cache.path = [ local-nix-cache.nix ];

  services.avahi.enable = true;
  services.avahi.publish.enable = true;
  services.avahi.publish.userServices = true;

}

Related work

To make this work I needed rust bindings to Nix which in turn required some stable-ish C bindings.

libnixstore-c - the C library to Nix. Exposing a very limited set of features.
libnixstore-sys - Rust bindings to the above C library.

andir / local-nix-cache