Something like virtualenv-clone

[HAOYU LI]

Dose pyenv or pyenv-virtualenv have a plugin like virtualenv-clone that could duplicate the virtual environment?

[native-api]

AFAICS, you can just copy the virtual environment's directory.

And create a symlink to it in $PYENV_ROOT/versions if needed.

[Alex Hedges]

I don't think that would be sufficient because all entry points in bin have a shebang with the absolute path to the python binary in the virtual environment. Just moving the directory doesn't work:

$ pyenv virtualenv 3.10 test
$ realpath $(pyenv prefix test)
/Users/ahedges/.pyenv/versions/3.10.9/envs/test
$ cd $(realpath $(pyenv prefix test))/../
$ head -1 test/bin/pip
#!/Users/ahedges/.pyenv/versions/3.10.9/envs/test/bin/python3.10
$ test/bin/pip --version
pip 22.3.1 from /Users/ahedges/.pyenv/versions/3.10.9/envs/test/lib/python3.10/site-packages/pip (python 3.10)
$ mv test test_moved
$ test_moved/bin/pip --version
-bash: test_moved/bin/pip: cannot execute: required file not found

You might be able to get it to work by changing all the shebangs, but there might be something else that can cause problems. For example, there is no guarantee that packages are written to support relocation.

[native-api]

In any case, the cloning boils down to smth like:

(pyenv shell $old; pip freeze >modules.lst)
pyenv virtualenv $new
(pyenv shell $new; pip install -f modules.lst)
rm modules.lst

One can easily adapt https://github.com/pyenv/pyenv-pip-migrate/blob/master/bin/pyenv-migrate to do that.

---

What's your use case anyway? I can hardly imagine why cloning a virtualenv would be such a frequent operation that it needs a dedicated command.
If e.g. you want to make a number of environments for Tox, it creates them itself: #21

[Alex Hedges]

In any case, the cloning boils down to smth like:

(pyenv shell $old; pip freeze >modules.lst)
pyenv virtualenv $new
(pyenv shell $new; pip install -f modules.lst)
rm modules.lst

That unfortunately doesn't work either. My laptop died last October, and I only had the output of pip freeze for all my environments in my backups. I encountered multiple problems with the restoration:

• pip does not store what index a package is downloaded from, so one needs to record indices used, for example, when installing versions of torch not hosted on PyPI.
• pip freeze does correctly output packages installed from the local filesystem, but that doesn't help when recreating an environment from scratch if you did not record the directory where the environment was created.
• Some of my environments were created under macOS 10.15, and they would not reproduce without setting SYSTEM_VERSION_COMPAT=1. However, setting it globally caused problems for environments that were created under macOS 11, so I needed to figure out whether it should be set for each environment.

These are just the ones I can think of off the top of my head. I believe I've encountered other problems as well (e.g., installation depended on what version of gcc is in PATH), but I don't remember more details for those situations.

[native-api]

If Virtualenv and Pip don't store some information that is required to create an environment, a would-be command won't be able to conjure that information from thin air, either.

[HAOYU LI]

In any case, the cloning boils down to smth like:

(pyenv shell $old; pip freeze >modules.lst)
pyenv virtualenv $new
(pyenv shell $new; pip install -f modules.lst)
rm modules.lst

One can easily adapt https://github.com/pyenv/pyenv-pip-migrate/blob/master/bin/pyenv-migrate to do that.

What's your use case anyway? I can hardly imagine why cloning a virtualenv would be such a frequent operation that it needs a dedicated command. If e.g. you want to make a number of environments for Tox, it creates them itself: #21

actually I think it very necessary to have the feature that allow users could clone or copy other virtual environment in pyenv-virtualenv. Thinking about an example like this, if you already have a virtual environment which contains pytorch or other big modules or frames, and now you have to build another virtual environment for another project which also depend on pytorch but exclude other module in previous virtual environment. These two different environments just like create a new branch in git. In this situation, why would I install the pytorch or other big modules from Internet again? It wastes more time than copy or clone from another envrionment.

[Alex Hedges]

Thinking about an example like this, if you already have a virtual environment which contains pytorch or other big modules or frames, and now you have to build another virtual environment for another project which also depend on pytorch but exclude other module in previous virtual environment. These two different environments just like create a new branch in git. In this situation, why would I install the pytorch or other big modules from Internet again? It wastes more time than copy or clone from another envrionment.

pip caches downloads, so you shouldn't need to download a package from the internet a second time unless you've cleared your cache.

As for "cloning" a virtual environment, it's not something that can be properly done within the current Python packaging ecosystem and would be very difficult to implement. If the feature was added to, for example, virtualenv, then it would make sense to add support in pyenv-virtualenv, but as it stands, the behavior you are proposing seems extremely out of this project's scope.