Per-virtualenv environment variables
michaeljones opened this issue · comments
I love the support for environment variables but I would like to be able specify particular environment variables for particular virtual envs, which doesn't seem to be possible from the config format.
Happy to submit patches if desirable!
I want at least to slow-cook the design before going to code. Can you give me an example use case or two? There is precedent for what you ask e.g. in tox.ini. But I guess which approach will be sane depends on what it's really for. Also, this adds an additional file read at startup so I want to know what we're paying for and whether it could reasonably be done without baking it in.
To me it feels weird for this one global file to be magically coupled to specific virtualenvs by name, they'd very easily get out of sync. I created vexrc for things like globally using certain switches, which is why it doesn't seem suited for this... and as you suggest, I don't want to shave the yak of the config format or introduce new concepts into it any more than I have to.
I think this might make more sense with some notion of a local/project-specific .vexrc.
Someone on HN mentioned virtualfish activating a virtualenv when you change into a directory, I said I can't do that but maybe I can change into a directory when you activate a virtualenv (retaining the desirable one-step-to-start feature).
In order to support that kind of thing I would need some place associated with the virtualenv to store the path it wants changed to, which could also have a set of environment variables as you expect. Anyway, apart from each specific feature it's worth thinking about.
Perhaps it could be a file put in the virtualenv to avoid an additional 'project' abstraction, it's a very natural correspondence and would also work with --path.
I'm less sure on how the data gets in, since the virtualenv is liable to be blown away; maybe the file could be generated from options or copied from a template? It feels awkward but maybe it's worth it.
I want there to be a more elegant solution than that but I have no more ideas right now.
Let's keep discussing this.
Thanks for the response. I'm not sure I can comment on the best way to achieve this. Per-project files perhaps. I agree that it would be a shame for entries in the config file to fall out of sync with the names of the virtualenvs.
My use case is that I have a django project. I understand it is best practice to have 'project secrets', ie. passwords, etc, managed through environment variables rather than checked into source control so I have specific environment variables I'd like to associate with that project. I'd rather not have such sensitive variables in my environment when I'm doing unrelated tasks.
I appreciate your followup, it helps me a lot to think about the proposal. My opinion today leans toward "no." Though encountering use cases might eventually persuade me.
In a way, this proposal is about using vex to augment virtualenv to be more like a container format, moving it in the direction of heroku/Foreman/Procfile or Docker/Dockerfile. Patching missing features into virtualenv itself is not an ambition I have for vex; vex is just a front end to activate functionality that's already in virtualenv; virtualenv is just a way to manage Python module dependencies. I don't want vex to have per-virtualenv parameters outside what virtualenv itself already defines. Even more than virtualenv, I want vex to have one main job and for fancier things to be achieved by combining with other tools.
Even with project scope set aside, I dislike making vex effectively a new virtualenv format; even if other tools can activate a vex-flavored virtualenv, this necessarily means dropping features which can break projects that depended on the vex-specific features to run. It should always be an option to use source bin/activate, or directly run virtualenv's python, or even use virtualenvwrapper or ave or invewrapper or whatever, interoperably with vex. Since virtualenv itself does not provide any way to populate environment variables as you ask, there is no interoperable way of adding this feature, which means to implement it at all encourages implicit project dependencies on vex itself since they'll break without vex around to read special files and propagate their contents into the environment.
As for your use case, I'll observe that using environment variables to provide secrets doesn't settle the issue of where they come from, which is usually a local file that has to be kept out of public version control anyway (.bashrc if nothing else). So why not just read the secrets out of files that are listed in .gitignore or stored outside the project tree? Or if you do need to use environment variables, why not use other tools to read/generate them and just let the values propagate through vex into the app environment?
$ A=42 vex cutlass bash -c 'echo $A'
42
Thanks for the response. I can see your reasoning and support you in taking your project in the direction you'd like. I'd raise a couple of points as I am a little confused:
- vex already supports environment variables in a non-virtualenv compatible manner. The .vexrc isn't honoured by other projects. I'm not sure why the
env:section is supported in there if you're concerned by that. - When using simply
source bin/activateI would added these environment variables to the end ofactivate, which I assume is the generalvirtualenvworkflow for this situation. I could store them in a gitignored-file but I would then want to source that fromactivateanyway just to smooth the workflow. I've reached for the .vexrc as vex doesn't seem to runactivateso I felt I'd benefit from another way of achieving the same results.
It is possible that I'm after a slightly different mix of functionality than you are, which is cool. Thanks for writing vex, I approve of the subshell idea for virtualenvs.
These are points worth thinking about, and I've thought about them, but I already write too much.
I only made vexrc so people wouldn't feel compelled to write a shell wrapper around it providing the right switches, and so I could do things like defining a default shell to use when no command was given, to give an interface similar to workon. I only added env to facilitate interactive shell configuration like prompts, which is a possible pain point for vex because not being sourced by the shell, it cannot directly munge shell prompts, and I would really like to avoid writing to people's bashrc files, or generating them.
The current env section does not define per-virtualenv parameters, so vexrc isn't offering any more power or encouragement to couple than you already get by editing bashrc, and it isn't any closer to a container format (or to tox!) than pip install --user is. The same abuses will be committed by editing bashrc, I can't stop that. But deprecating env on this account could hurt vex's prompt story for no real reason. I figure the mess of editing bashrc or equivalent with a million PROJECT_A_SETTING_FOO=1;PROJECT_B_SETTING_BAR=2 is sufficient deterrent; you yourself are discouraged from using it, for example.
vex's role in this kind of error is passive at worst, no document or command line switch is there to suggest it. Only once I add formal switches in vex to set per-virtualenv parameters (exactly like in heroku cli) and it is not an accidentally possible hack but a first-class feature that's clean enough for a sane person like you to want to use, am I actively encouraging coupling between that project file (hence vex) and the project, in any way that bash isn't already passively doing.
I'm just not persuaded of the strength of the use cases (yet?) Because I think your use case is solidly addressable with the normal little things like .gitignore on a secret file or launch script. There is really nothing to uniquely identify vex as the right home of this responsibility. The decision to communicate config to the app through global environment variables does nothing particularly to enhance security, it just delegates the question of how to discover and read config to some other component. That other component should be part of your web server config, or web framework, or custom launcher script, or startup code in the application itself - but regardless of where you put it, it should be in source control, because your application can't run without it. I could write launcher code for your project and put it in vex so you don't have to put it in your project, but I don't see the advantage. Anyway, you still have the issue of securing the data at rest. As for the actual secret keys, you can use any technique for safely storing in your filesystem or database or whatever.
By the time we are writing using a CLI to maintain config files with environment variables in them, why not tox or docker? We are properly talking about launching processes inside some kind of container. That is something that can be built on top of virtualenv. But it isn't something that I want to use vex to build on top of virtualenv.
On the other hand, by the time you are editing bin/activate (and losing and repeating your edits every time a virtualenv has to be blown away, because they're meant to be disposable) and you absolutely need per-virtualenv customization which requires a significant number of the features of shell, the ideal solution for you is virtualenvwrapper hooks. Not even reasonably simple shell functions will suffice for you. There's no way vex can compete with software built in shell for arbitrarily customizable integration with shell, if you believe that's a requirement.
vex really only has a niche if it's a simple and acceptably fast interface for running something in a virtualenv. And that still leaves plenty of scope for composition with other tools. There might be another project in this which finds config and puts it in the environment for you.
It looks like you understand this much better than I do. Thanks for the explanation. Much to think about!
I really doubt that - maybe I'm too thick to see it yet, but thanks for the suggestion and the discussion!