Subprocess spawning
ionelmc opened this issue · comments
How come vex is using Popen to run the subprocess (instead of os.execvp or something) on *nix? (not on Windows obviously 😁)
This will invariably create problems with signal handling and getting a handle to the actual pid ...
I'm interested in hearing why subprocess.Popen was used over os.execvp.
Do you have a specific problem or use case to look at?
Eg: using a supervisor. Supervisor sends SIGTERM but the supervisor only knows the parent process. Vex doesn't forward signals to the child.
vex is designed for interactive use. I made it because I was tired of the scrunginess of virtualenvwrapper modifying the shell environment in-place and I wanted a different interface. For that purpose, it is already working.
For non-interactive uses, I think people who are hungry for control will find it better in several ways to just run the virtualenv's python directly. Fewer layers, less overhead, and you always provide the absolute path. The feature of having virtualenvs in ~/.virtualenvs is not so useful for daemons.
So I'm not doing this, but I want to explain why and it isn't because you are wrong!
On one hand, I don't see a lot of real uses. But on the other hand, this means taking something that already works for me and a few other people, and changing it a lot, which will take me an exaggeratedly long time to do without introducing regressions, totally out of proportion to the benefit anyone will get out of it. Signal interactions are basically nondeterministic and properly testing them is hell. I can't wait for the Github issues when users encounter a subtle corner case that doesn't even have a traceback attached to it. Then there are cross-platform issues, which I have found to be really unenjoyable as they touch signals. I can relate to wanting execvp behavior as a feature but I don't know if you considered what it takes to support this behavior on Windows, and it adds support burden for me to try to supply separate implementations for the two platforms and try to make them act the same. That is essentially what subprocess is already. And I have already gotten angry flak for supposedly not supporting Windows enough, so it seems bad to intentionally do things that are likely to cause even more of that.
To me, this kind of thing is trying to write C in Python. At the C level, Python itself looks huge and messy, dynamic typing creates insane amounts of indirection, virtualenv is a total absurdity and maybe even subprocess looks dumb. I can understand this kind of perspective. But at the Python level, where I live these days, I don't care about having an extra process in the process tree, and os.execvp is really an ancient and unsafe interface with platform issues, while I just want normal things to work well and simply.