Permission denied error when building symlink derivation

Question

Permission denied error when building symlink derivation

rvl opened this issue 9 months ago · comments

Describe the bug

My apologies if this is already fixed on master branch, but I couldn't identify any particular issue or PR which describes this issue.

Nix 1.19.2 fails to build a derivation containing a symlink to store path.

# test-symlink.nix
{ local ? "/home/rodney/ops/nixpkgs"
, pkgs ? import local {}
}: rec {
  direct-symlink = pkgs.runCommand "direct-symlink" {} ''
    ln -vs ${local}/.version $out
  '';
  indirect-symlink = pkgs.runCommand "indirect-symlink" {} ''
    ln -vs ${direct-symlink} $out
  '';
}

Steps To Reproduce

The error message is:

rodney@tethys:~/ops/nixpkgs % nix build --experimental-features "nix-command" -f test-symlink.nix && readlink -f result*
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled
error:
       … while setting up the build environment

       error: getting attributes of path '/nix/store/ws9yl6ph10v79gx4p9ilhyxg214xf7i7-direct-symlink': Permission denied

Expected behavior

I expected both derivations to build and for the resulting store paths to be resolvable symlinks.

This is what happens with nix-2.18.1:

rodney@tethys:~/ops/nixpkgs % nix build --experimental-features "nix-command" -f test-symlink.nix && readlink -f result*
warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring
/home/rodney/ops/nixpkgs/.version
/home/rodney/ops/nixpkgs/.version

nix-env --version output

This is the nixUnstable package, corresponding to revision 2c7f3c0fb7c08a0814627611d9d7d45ab6d75335 of nixpkgs.

nix (Nix) 2.19.2

Additional context

Introduced by

#8965

You might ask, why build such a silly derivation? Well, it's how mkOutOfStoreSymlink works under home-manager.

See: nix-community/home-manager#4692

Priorities

Add 👍 to issues you find important.

nixos-discourse · Answer 1 · Tue Dec 19 2023 10:59:14 GMT+0800 (China Standard Time)

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-12-18-nix-team-meeting-minutes-113/37050/1

nixos-discourse · Answer 2 · Sat Dec 23 2023 11:17:34 GMT+0800 (China Standard Time)

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/home-manager-mkoutofstoresymlink-and-nixunstable/37241/1

Silvan Mosberger · Answer 3 · Sat Dec 23 2023 22:13:57 GMT+0800 (China Standard Time)

I tried to reproduce this, but struggled. I wrote a NixOS test for this that shows no problems:

# nixos-test.nix
let
  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/d6863cbcbbb80e71cecfc03356db1cda38919523";
  pkgs = import nixpkgs {
    config = {};
    overlays = [];
    system = "x86_64-linux";
  };
in
pkgs.nixosTest {
  name = "test";
  skipTypeCheck = true;
  nodes.machine = {
    environment.etc.nixpkgs.source = nixpkgs;
    system.extraDependencies = [
      (pkgs.closureInfo {
        rootPaths = [ pkgs.stdenvNoCC.drvPath ];
      })
    ];
    nix.package = pkgs.nixUnstable;
    environment.etc."test-symlink.nix".text = ''
      # test-symlink.nix
      { local ? "/etc/nixpkgs"
      , pkgs ? import local {}
      }: rec {
        direct-symlink = pkgs.runCommand "direct-symlink" {} '''
          ln -vs ''${local}/.version $out
        ''';
        indirect-symlink = pkgs.runCommand "indirect-symlink" {} '''
          ln -vs ''${direct-symlink} $out
        ''';
      }
    '';
  };

  testScript = ''
    start_all()
    machine.succeed('nix-build /etc/test-symlink.nix')
    print(machine.succeed('nix-env --version'))
    print(machine.succeed('readlink -f result*'))
  '';
}

Run with

$ nix-build nixos-test.nix
[...]
machine # these 2 derivations will be built:
machine #   /nix/store/lqh27yh50v8zyv57cwk4l5zz2yg8046q-direct-symlink.drv
machine #   /nix/store/4p887hcx1bks8h65p0j3ghfnximkddyf-indirect-symlink.drv
machine # warning: Ignoring setting 'auto-allocate-uids' because experimental feature 'auto-allocate-uids' is not enabled
machine # warning: Ignoring setting 'impure-env' because experimental feature 'configurable-impure-env' is not enabled
machine # building '/nix/store/lqh27yh50v8zyv57cwk4l5zz2yg8046q-direct-symlink.drv'...
machine # '/nix/store/96664m3zyjnw2xyhpkyipxcmf5s7vv3w-direct-symlink' -> '/etc/nixpkgs/.version'
machine # building '/nix/store/4p887hcx1bks8h65p0j3ghfnximkddyf-indirect-symlink.drv'...
machine # '/nix/store/hq63b5zkir3ibsas5w9rkyp8gllgs6jx-indirect-symlink' -> '/nix/store
/96664m3zyjnw2xyhpkyipxcmf5s7vv3w-direct-symlink'
(finished: must succeed: nix-build /etc/test-symlink.nix, in 11.68 seconds)
machine: must succeed: nix-env --version
(finished: must succeed: nix-env --version, in 0.06 seconds)
nix-env (Nix) 2.19.2

machine: must succeed: readlink -f result*
(finished: must succeed: readlink -f result*, in 0.02 seconds)
/nix/store/61gw1axgxrvx0bnxv2dla6zhkgwv45zr-source/.version
/nix/store/61gw1axgxrvx0bnxv2dla6zhkgwv45zr-source/.version

(finished: run the VM test script, in 11.93 seconds)
test script finished in 11.96s
cleanup
kill machine (pid 7)
machine # qemu-kvm: terminating on signal 15 from pid 4 (/nix/store/5k91mg4qjylxbfvrv748smfh51ppjq0g-python3-3.11.6/bin/python3.11)
(finished: cleanup, in 0.05 seconds)
kill vlan (pid 5)
/nix/store/7dwclnxw0xdvmqvrmi19qyyxqs1xwv03-vm-test-run-test

I don't have more time right now, but if somebody could adjust this NixOS test to show the problem, that would be great!

Rodney Lorrimar · Answer 4 · Sun Dec 24 2023 22:03:10 GMT+0800 (China Standard Time)

I don't have more time right now, but if somebody could adjust this NixOS test to show the problem, that would be great!

Here @infinisil, try this one:

# nixos-test.nix
let
  nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/tarball/d6863cbcbbb80e71cecfc03356db1cda38919523";
  pkgs = import nixpkgs {
    config = {};
    overlays = [];
    system = "x86_64-linux";
  };

  user = "alice";
  target = "/home/${user}/file";
in
pkgs.nixosTest {
  name = "test";
  skipTypeCheck = true;
  nodes.machine = {
    system.extraDependencies = [
      (pkgs.closureInfo {
        rootPaths = [ pkgs.stdenvNoCC.drvPath ];
      })
    ];
    nix.package = pkgs.nixUnstable;
    nix.nixPath = ["nixpkgs=${nixpkgs}"];
    environment.etc."test-symlink.nix".text = ''
      { pkgs ? import <nixpkgs> {} }: let
        direct-symlink = pkgs.runCommand "direct-symlink" {} '''
          ln -vs ${target} $out
        ''';
        indirect-symlink = pkgs.runCommand "indirect-symlink" {} '''
          ln -vs ''${direct-symlink} $out
        ''';
      in
        indirect-symlink
    '';
    users.users.${user}.isNormalUser = true;
    systemd.tmpfiles.rules = ["f ${target} 0644 ${user} ${user} - test"];
  };

  testScript = ''
    start_all()
    print(machine.succeed('nix --version'))
    machine.succeed("su -l ${user} -c 'nix-build /etc/test-symlink.nix'")
    machine.succeed("su -l ${user} -c 'readlink -f result'")
    machine.succeed("su -l ${user} -c 'test `readlink -f result` = ${target}'")
  '';
}

I believe that the three necessary conditions for failure are:

Symlink target is outside of /nix/store
nix-build is invoked from a normal user, not root.
nix.package = pkgs.nixUnstable;

Silvan Mosberger · Answer 5 · Mon Jan 01 2024 07:14:57 GMT+0800 (China Standard Time)

Thanks, using bisection I was able to confirm that #8965 (as already suspected by @cole-h) indeed is the culprit for this regression.

John Ericson · Answer 6 · Tue Jan 09 2024 02:49:09 GMT+0800 (China Standard Time)

When it does work like, what do the references look like? Is it possible the hardlinking was incorrectly following the symlink so we referencing paths not in our closure?!

Artturin · Answer 7 · Tue Jan 09 2024 05:26:43 GMT+0800 (China Standard Time)

Rudimentary workaround/fix #9723

nixos-discourse · Answer 8 · Fri Jan 12 2024 16:34:26 GMT+0800 (China Standard Time)

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-01-08-nix-team-meeting-minutes-114/38156/1

nixos-discourse · Answer 9 · Thu Mar 28 2024 10:20:45 GMT+0800 (China Standard Time)

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/an-additional-nix-installed-by-home-manager/42298/1

Siddhartha Sahu · Answer 10 · Mon Apr 01 2024 02:01:12 GMT+0800 (China Standard Time)

I was just wondering: is this bug not considered a breaking change critical enough to be prioritized? Stable nix releases > 2.18 breaks home-manager users as per this bug.

Does the behavior being relied upon by home-manager not come under stability guarantees? Asking because the discourse report marked this as Annoying but not critical.

Rodney Lorrimar · Answer 11 · Mon Apr 01 2024 10:29:51 GMT+0800 (China Standard Time)

For me personally, it's a critical issue because my home-manager config which used to work nicely is now broken.

However I think home-manager could work around the issue by building symlinks within a nix store path directory, rather than as top-level nix store paths. So from that perspective it's "annoying but not critical."

For stability guarantees I guess we would need to know which behaviours (if not all) should be stable, and what does correct behaviour look like. Perhaps this is the more important issue. What is the meaning of a symlink at the top level of a nix store?

alois31 · Answer 12 · Tue Apr 09 2024 15:46:48 GMT+0800 (China Standard Time)

Not only does this lead to "permission denied" errors when the symlink target is not accessible to the build user, it also introduces a huge impurity when it is.

srestegosaurio · Answer 13 · Wed Apr 10 2024 00:48:10 GMT+0800 (China Standard Time)

I don't know but impermanence is pretty broken for me right now and by being in a root-on-tmpfs system this is quite critical.