tox-conda interacts badly with numpy on Windows, Python 3.7/3.8
goerz opened this issue · comments
Having numpy in conda_deps on Windows with Python 3.7 or Python 3.8 causes a crash when any command inside the environment tries to import numpy. Python 3.6 works.
See cross-reference numpy/numpy#15537
It seems the issue may be that numpy does not load unless the conda environment is explicitly activated (see numpy/numpy#15537 (comment)). Does tox-conda activate the environments that it manages (or mimick exactly what conda activate <envname> would do)?
I believe we are also seeing this issue, however we are seeing it in a tox environment that has basepython=python3.6.
I also believe this is due to tox-conda not activating the conda env when it runs the commands, because if I activate the environment that it creates and then run the commands manually in there it works fine.
I think I might be seeing this as well (though for me it fails on 3.6 as well). Same setup as @goerz mentioned, numpy gets installed via conda_deps. Only fails on windows. In case it's an important detail, I'm on github actions, using goanpeca/setup-miniconda@v1.6.0 to install conda. Here's a failing test: https://github.com/tlambert03/napari-omero/runs/804849289?check_suite_focus=true#step:6:62
I have managed to find a work around for this. By setting CONDA_DLL_SEARCH_MODIFICATION_ENABLE = 1 in the setenv section of the tox testenv configuration it works again. There is more information in the conda docs about this workaround.
A detail about @Choc13 's DLL env variable fix. I've noticed this only works with numpy from the default channel from anaconda.com. It does not however, seem to work if you install numpy from conda-forge.
@rwhitt2049, I can indeed confirm that if does not work with NumPy installed from conda-forge. It makes sense that it doesn't work because this is what I read in the conda docs:
Anaconda has built special patches into its builds of Python to enable this functionality. If you get your Python package from somewhere else (e.g. conda-forge), these flags may not do anything.
I still see this issue for NumPy v1.20.1.
Did you manage to find a workaround?
I managed to work around it by commenting out the
conda_channels =
conda-forge
section and having setenv = CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1.
Unfortunately, this led to some other issues, like this. In the end I had to install both numpy, scipy, and h5py from the defaults channel to make it work.
I am looking forward to a better (more stable) solution!
@basnijholt, I got around it by setting my conda channels as follows:
conda_channels=
defaults
conda-forge
... and then adding an env variable as per Choc13's comment above:
setenv = CONDA_DLL_SEARCH_MODIFICATION_ENABLE=1
I have dependencies on conda-forge as well, which is why it's still in my conda_channels list. So functionally this will pull dependencies from defaults first, and then check conda-forge if it can't find the package there.
Setting the env variable took care of my dll issues.
tox-conda 0.8 should address this kind of issue (although not perfectly), you may give it a try.
No update from author: closing.