jetpack-io / devbox

Current Behavior (bug)

My fish config contains:

set EXA_STANDARD_OPTIONS --group --header --group-directories-first --icons

but exportify:

devbox/internal/devbox/envvars.go

Lines 17 to 22 in 2fee55b

    
           // exportify formats vars as a line-separated string of shell export statements. 
        
           // Each line is of the form `export key="value";` with any special characters in 
        
           // value escaped. This means that the shell will always interpret values as 
        
           // literal strings; no variable expansion or command substitution will take 
        
           // place. 
        
           func exportify(vars map[string]string) string {

exports every env variable enclosed with "". This results in the incorrect behaviour: a variable exported with " will be treated as a single argument, while a variable without " will correctly treat the values as separate arguments. In fish, all variables are lists, so it's not always correct to enclose variables with strings.

The $EXA_STANDARD_OPTIONS var is sent to exa and when exported with ", exa/eza treats the whole value as a single argument, returning an error: eza: Unknown argument --group --header --group-directories-first --icons. If exported without ", it works as expected, because fish treats those as an argument list.

Expected Behavior (fix)

Maintain semantics of user-exported env variables. Or, even better, when using fish, do not re-export env variables that are not relevant to devbox.

If this comment works as intended, there's no need to reexport something that was correctly exported in my fish config:

devbox/internal/devbox/shellrc_fish.tmpl

Lines 6 to 9 in 2fee55b

    
           It does _not_ include the user's original fish config, because unlike other 
        
           shells, fish has multiple files as part of its config, and it's difficult 
        
           to start a fish shell with a custom fish config. Instead, we let fish read 
        
           the user's original config directly, and run these commands next.

Additional context

$ devbox version -v
Version:     0.8.7
Platform:    darwin_arm64
Commit:      525bb17ed665bb3066fbd3fb9bccec0aa961ecb7
Commit Time: 2024-01-23T19:15:31Z
Go Version:  go1.21.5
Launcher:    0.2.1

@arthurgeek thanks for filing this issue. I tried to repro it but was having some diffculty. Could you point out what I should do differently?

Here's what I did:

# set EXA-STANDARD_OPTIONS in config.fish:
> cat ~/.config/fish/config.fish
...
set EXA_STANDARD_OPTIONS --group --header --group-directories-first --icons

# add eza to devbox global
> devbox global add eza

# verify the standard options are set
(devbox) > echo $EXA_STANDARD_OPTIONS
--group --header --group-directories-first --icons

# run eza, which works fine by listing the files in my directory
> eza -al

(related: I did run into a few fish-related bugs: #1755 and #1756)

@savil thanks for looking into it!

sorry, I should have been more explicit: $EXA_STANDARD_OPTIONS is not directly used by eza. You have to call the binary with it. I don't have access to a machine installed with devbox right now, but this should give you the exact same behavior:

$ set EXA_STANDARD_OPTIONS --group --header --group-directories-first --icons
$ echo $EXA_STANDARD_OPTIONS
--group --header --group-directories-first --icons
$ eza $EXA_STANDARD_OPTIONS
[works, listing files/directories]
$ export EXA_STANDARD_OPTIONS="--group --header --group-directories-first --icons"
$ echo $EXA_STANDARD_OPTIONS
--group --header --group-directories-first --icons
$ eza $EXA_STANDARD_OPTIONS
eza: Unknown argument --group --header --group-directories-first --icons
$

Note: I'm using export in my example because that's the command devbox runs.

Thanks! I can now reproduce it. Looked into it a bit today.

I tried modifying our implementation to not wrap the values in quotations, but that breaks devbox shell because some env-vars produced by nix print-dev-env have values that really do need the double-quote wrapping.

Will look into alternate fixes...

I tried modifying our implementation to not wrap the values in quotations

I don't think this is a good approach. devbox shouldn't make any assumption whether a variable is a list with multiple items (without quotations) or not. Re-exporting everything without quotations could also break where a user variable is expected to be a single-item list.

I'm not familiar with devbox's code or nix, but I think the best alternative is to not copy any variable from the current env that's not relevant to devbox. If a variable it's not used by devbox, then don't copy/re-export it. This way you have full control: you know if a variable should be a list or not.

From what I looked at the code, it copies everything because for other shells when opening devbox shell you need to ensure all the vars from current env are present, to keep a consistent behavior with current env. But since for fish you actually load the user config when running devbox shell, you don't need to re-export everything, you can simply export only the variables that are relevant to devbox.

I think you could modify this behavior when in fish:

devbox/internal/devbox/devbox.go

Lines 769 to 795 in 5585d21

    
           // computeEnv computes the set of environment variables that define a Devbox 
        
           // environment. The "devbox run" and "devbox shell" commands source these 
        
           // variables into a shell before executing a command or showing an interactive 
        
           // prompt. 
        
           // 
        
           // The process for building the environment involves layering sets of 
        
           // environment variables on top of each other, with each layer overwriting any 
        
           // duplicate keys from the previous: 
        
           // 
        
           //  1. Copy variables from the current environment except for those in 
        
           //     ignoreCurrentEnvVar, such as PWD and SHELL. 
        
           //  2. Copy variables from "nix print-dev-env" except for those in 
        
           //     ignoreDevEnvVar, such as TMPDIR and HOME. 
        
           //  3. Copy variables from Devbox plugins. 
        
           //  4. Set PATH to the concatenation of the PATHs from step 3, step 2, and 
        
           //     step 1 (in that order). 
        
           // 
        
           // The final result is a set of environment variables where Devbox plugins have 
        
           // the highest priority, then Nix environment variables, and then variables 
        
           // from the current environment. Similarly, the PATH gives Devbox plugin 
        
           // binaries the highest priority, then Nix packages, and then non-Nix 
        
           // programs. 
        
           // 
        
           // Note that the shellrc.tmpl template (which sources this environment) does 
        
           // some additional processing. The computeEnv environment won't necessarily 
        
           // represent the final "devbox run" or "devbox shell" environments. 
        
           func (d *Devbox) computeEnv(ctx context.Context, usePrintDevEnvCache bool) (map[string]string, error) {

perhaps even here:

devbox/internal/devbox/devbox.go

Lines 1220 to 1222 in 5585d21

    
           // parseEnvAndExcludeSpecialCases converts env as []string to map[string]string 
        
           // In case of pure shell, it leaks HOME and it leaks PATH with some modifications 
        
           func (d *Devbox) parseEnvAndExcludeSpecialCases(currentEnv []string) (map[string]string, error) {

If that's not feasible, then another suggestion would be to use fish's count:

$ set EXA_STANDARD_OPTIONS --group --header --group-directories-first --icons
$ count $EXA_STANDARD_OPTIONS
4
$ set EXA_STANDARD_OPTIONS "--group --header --group-directories-first --icons"
$ count $EXA_STANDARD_OPTIONS
1
$

and re-export with/without quotations depending on the count value.

I don't see any other way to keep variable "semantics". In both cases you're adding code that's specific only to fish, but the first option might lead to less problems down the road, as you're not trying to touch anything that's outside the scope of devbox.

	// exportify formats vars as a line-separated string of shell export statements.
	// Each line is of the form `export key="value";` with any special characters in
	// value escaped. This means that the shell will always interpret values as
	// literal strings; no variable expansion or command substitution will take
	// place.
	func exportify(vars map[string]string) string {

	It does _not_ include the user's original fish config, because unlike other
	shells, fish has multiple files as part of its config, and it's difficult
	to start a fish shell with a custom fish config. Instead, we let fish read
	the user's original config directly, and run these commands next.

	// computeEnv computes the set of environment variables that define a Devbox
	// environment. The "devbox run" and "devbox shell" commands source these
	// variables into a shell before executing a command or showing an interactive
	// prompt.
	//
	// The process for building the environment involves layering sets of
	// environment variables on top of each other, with each layer overwriting any
	// duplicate keys from the previous:
	//
	// 1. Copy variables from the current environment except for those in
	// ignoreCurrentEnvVar, such as PWD and SHELL.
	// 2. Copy variables from "nix print-dev-env" except for those in
	// ignoreDevEnvVar, such as TMPDIR and HOME.
	// 3. Copy variables from Devbox plugins.
	// 4. Set PATH to the concatenation of the PATHs from step 3, step 2, and
	// step 1 (in that order).
	//
	// The final result is a set of environment variables where Devbox plugins have
	// the highest priority, then Nix environment variables, and then variables
	// from the current environment. Similarly, the PATH gives Devbox plugin
	// binaries the highest priority, then Nix packages, and then non-Nix
	// programs.
	//
	// Note that the shellrc.tmpl template (which sources this environment) does
	// some additional processing. The computeEnv environment won't necessarily
	// represent the final "devbox run" or "devbox shell" environments.
	func (d *Devbox) computeEnv(ctx context.Context, usePrintDevEnvCache bool) (map[string]string, error) {

	// parseEnvAndExcludeSpecialCases converts env as []string to map[string]string
	// In case of pure shell, it leaks HOME and it leaks PATH with some modifications
	func (d *Devbox) parseEnvAndExcludeSpecialCases(currentEnv []string) (map[string]string, error) {

[Bug]: export of fish shell environment variables do not always works as intended