PATH variable keeps growing in nested shells

[charego]

Thank you for this plugin.

I've just noticed that in nested shells, the PATH variable continues to grow.

> set --show PATH | grep nix
$PATH[6]: |/Users/charego/.nix-profile/bin|
$PATH[7]: |/nix/var/nix/profiles/default/bin|

> exec fish
> set --show PATH | grep nix
$PATH[1]: |/Users/charego/.nix-profile/bin|
$PATH[2]: |/nix/var/nix/profiles/default/bin|
$PATH[14]: |/Users/charego/.nix-profile/bin|
$PATH[15]: |/nix/var/nix/profiles/default/bin|

> exec fish
> set --show PATH | grep nix
$PATH[1]: |/Users/charego/.nix-profile/bin|
$PATH[2]: |/nix/var/nix/profiles/default/bin|
$PATH[9]: |/Users/charego/.nix-profile/bin|
$PATH[10]: |/nix/var/nix/profiles/default/bin|
$PATH[16]: |/Users/charego/.nix-profile/bin|
$PATH[17]: |/nix/var/nix/profiles/default/bin|

I'm not sure if this should be considered a bug with nix-env.fish or with the script that nix-env.fish sources and replays. In my case, I'm running a multi-user install on MacOS and nix-env.fish is sourcing the nix-daemon.sh script.

Here is the relevant snippet from this plugin:

set -l nix_profile_path /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh
...
# Source the nix setup script
# We're going to run the regular Nix profile under bash and then print out a few variables
for line in (command env -u BASH_ENV bash -c '. "$0"; for name in PATH "${!NIX_@}"; do printf "%s=%s\0" "$name" "${!name}"; done' $nix_profile_path | string split0)
  set -xg (string split -m 1 = $line)
end

Here is the relevant snippet from nix-daemon.sh:

# Only execute this file once per shell.
if [ -n "${__ETC_PROFILE_NIX_SOURCED:-}" ]; then return; fi
__ETC_PROFILE_NIX_SOURCED=1

export NIX_PROFILES="/nix/var/nix/profiles/default $HOME/.nix-profile"
...
export PATH="$HOME/.nix-profile/bin:/nix/var/nix/profiles/default/bin:$PATH"

I'm able to "fix" the problem by updating nix-daemon.sh to check for duplicates:

# Prepend profile binaries to PATH if not already present.
for i in $NIX_PROFILES; do
  if [ -d "$i/bin" ] && [[ ":$PATH:" != *":$i/bin:"* ]]; then
    export PATH="$i/bin${PATH:+":$PATH"}"
  fi
done

But notice the script already has its own guard at the top, which doesn't work for us. So should this be guarded against by nix-env.fish, or is it something we need to push upstream? Thanks!

[Lily Ballard]

The script has its own guard at the top, but it doesn't export the guard and so fish cannot see it. This means subshells don't see the guard either. The script probably should be updated to export the guard, although nix-env.fish still wouldn't see it as it only grabs paths starting with NIX_. So nix-env.fish would also need to be updated to export the guard too. Ideally it would export anything with _NIX_ in the name but bash does not have any easy way to even find such variables (the best I've found is declare -p to print everything and grep that, but multiline variables screw up the grepping as bash includes the newlines verbatim instead of using e.g. $'' in the output).

Probably the best solution is to get an upstream patch for nix-daemon.sh that actually makes it check if the paths it wants are in PATH before adding them. If it does that, the unexported guard doesn't matter, and nix-env.fish will pick up the fix automatically (and I'd argue that the guard should then remain unexported so that way spawning a subshell with a modified PATH will reapply the Nix profile).

As a side note, NixOS (and nix-darwin) export their guard variables, but they're overriding a lot more of the environment and so suppressing that on subshells is a good idea. nix-daemon.sh on the other hand only sets Nix-specific variables and so rerunning that on subshells should be fine as long as it's idempotent (which appears to just require checking PATH).

[Lily Ballard]

I suppose nix-env.fish could export the guard variable itself, but I don't consider that a good solution, and it wouldn't work for single-user installs anyway as the single-user profile doesn't have a guard variable (unless nix-env.fish adds its own guard variable check too). I would much prefer to see this fixed upstream instead.

[Lily Ballard]

I filed this as NixOS/nix#5950.

[charego]

Thanks for this thoughtful response and quick action, much appreciated.

[James Hillyerd]

Wanted to add a note that this is causing problems for me in development shells. For example, I have nodejs 16.x in my profile via nix-env, if I enter a nix-shell (bash) for a project that requires nodejs 14.x via shell.nix, it works, but if I then enter fish, my .nix-profile/bin is added to the front of my path, and I'm back on 16.x. Should I add this to the nix 5950 bug?

[Lily Ballard]

@jhillyerd What happens if you run a nested bash shell? Does it also do that? It's been a while since I looked at the "normal" Nix install. IIRC it uses bashrc so interactive bash shells should indeed setup the environment again and therefore muck with PATH but it would be nice to confirm this (I'm assuming you've installed Nix on top of an existing OS, but if you are using NixOS then you really should use the NixOS fish module instead of this plugin).

If an interactive bash shell reproduces this (and your environment is still set up such that just running a new terminal with a bash login shell initializes Nix, i.e. /etc/bashrc hasn't been reset), then you should comment in NixOS/nix#5950 about this as any PATH deduplication that is done as a result of that ticket should make sure it handles this case properly.

If bash is set up for Nix correctly but running an interactive bash subshell does not have this problem, that would be good to know.

[David Nadlinger]

@lilyball I've just run into the same issue. If I nix develop, which uses bash by default, everything is fine, but if I run nix develop --command fish, I end up with the duplicate $HOME/.nix-profile/bin /nix/var/nix/profiles/default/bin at the beginning of the $PATH. This is irrespective of whether the parent shell calling nix develop is itself bash or fish.

[James Hillyerd]

@lilyball I use a mix of machines, some are NixOS. You are correct though, this particular one is WSL2 Ubuntu w/ Nix. Thanks for the heads up on the fish module, I'll check that one out for my NixOS box.

Testing:

1. running another bash inside of nix-shell keeps the correct PATH, it's exactly the same as it's parent shell.
2. bash --login prefixes /home/james/bin:/home/james/.nix-profile/bin onto PATH
3. fish prefixes /home/james/.nix-profile/bin /home/james/.fzf/bin

[Omar Änwar]

@jhillyerd, I can confirm, exploring the same kind of issue on wsl