v0.9.0 seems to break tox
WeatherGod opened this issue · comments
Not exactly sure what is going on, but it seems like the environment activation is going haywire. A snippet of my output with -vvv.
... [snipped] ...
py38 start: run-test-pre
py38 run-test-pre: PYTHONHASHSEED='4208903247'
py38 finish: run-test-pre after 0.00 seconds
py38 start: run-test
py38 run-test: commands[0] | pytest --cov=raster --cov-report=xml --cov-report=term-missing tests/
setting PATH=[snipped]/.tox/py38/bin:[snipped]:/home/broot/.local/bin
[684100] [snipped]$ [snipped]/.tox/py38/bin/pytest --cov=raster --cov-report=xml --cov-report=term-missing tests/
/tmp/tmp1g07361n: line 1: export: `.': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `[snipped]/.tox/py38/etc/conda/activate.d/gdal-activate.sh': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `.': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `[snipped]/.tox/py38/etc/conda/activate.d/geotiff-activate.sh': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `.': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `[snipped]/.tox/py38/etc/conda/activate.d/pdal-activate.sh': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `.': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `[snipped]/.tox/py38/etc/conda/activate.d/pdal-python-activate.sh': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `.': not a valid identifier
/tmp/tmp1g07361n: line 1: export: `[snipped]/.tox/py38/etc/conda/activate.d/proj4-activate.sh': not a valid identifier
... [snipped] ...
Downgrading to v0.8.4 makes everything work as it has before.
Could you please provide a reproducer?
As well as information on your system, conda info --all.
I can try. I am actually not terribly familiar with tox or tox-conda. I only noticed this because the CI/CD of our project broke today, and I didn't set any of this up. I guess you would want a tox.ini and a conda environment file, right?
Yes let's start with your tox.ini and the output of conda info --all.
Well, I made a minimized version of that:
(base) bash-4.4$ cat tox.ini
[tox]
envlist = py38
[testenv]
conda_spec = env.txt
commands = echo foobar
(base) bash-4.4$ cat env.txt
numpy
gdal
fiona
(base) bash-4.4$ cat setup.py
from setuptools import (
find_packages,
setup
)
setup(
name='foo',
description='Foo to the Bar',
package_dir={'foo': 'foo'},
packages=find_packages(where='foo', exclude='.'),
# Dependencies
install_requires=[
'numpy',
'fiona'
],
# pytest setup
setup_requires=['pytest-runner'],
tests_require=['pytest', 'pytest-cov']
)
(base) bash-4.4$ cat foo/__init__.py
import fiona
import numpy as np
This was done in a fairly clean conda environment (the base one, I pip-installed tox and tox-conda today in it).
(base) bash-4.4$ tox
GLOB sdist-make: /rd22/scratch/broot/Programs/example/setup.py
py38 create: /rd22/scratch/broot/Programs/example/.tox/py38
py38 installcondadeps: --file=/rd22/scratch/broot/Programs/example/env.txt
py38 inst: /rd22/scratch/broot/Programs/example/.tox/.tmp/package/1/foo-0.0.0.zip
py38 installed: attrs @ file:///tmp/build/80754af9/attrs_1620827162558/work,certifi==2021.10.8,click @ file:///home/linux1/recipes/ci/click_1610990599742/work,click-plugins @ file:///Users/ktietz/demo/mc3/conda-bld/click-plugins_1630665883852/work,cligj==0.7.2,Fiona==1.8.13.post1,foo @ file:///rd22/scratch/broot/Programs/example/.tox/.tmp/package/1/foo-0.0.0.zip,GDAL==3.0.2,mkl-fft==1.3.1,mkl-random @ file:///tmp/build/80754af9/mkl_random_1626186064646/work,mkl-service==2.4.0,munch @ file:///Users/ktietz/demo/mc3/conda-bld/munch_1630566382504/work,numpy @ file:///tmp/build/80754af9/numpy_and_numpy_base_1634095647912/work,Shapely @ file:///tmp/build/80754af9/shapely_1632813486044/work,six @ file:///tmp/build/80754af9/six_1623709665295/work
py38 run-test-pre: PYTHONHASHSEED='3078875635'
py38 run-test: commands[0] | echo foobar
WARNING: test command found but not installed in testenv
cmd: /usr/bin/echo
env: /rd22/scratch/broot/Programs/example/.tox/py38
Maybe you forgot to specify a dependency? See also the allowlist_externals envconfig setting.
DEPRECATION WARNING: this will be an error in tox 4 and above!
/tmp/tmpuu_9e2cq: line 1: export: `.': not a valid identifier
/tmp/tmpuu_9e2cq: line 1: export: `/rd22/scratch/broot/Programs/example/.tox/py38/etc/conda/activate.d/gdal-activate.sh': not a valid identifier
/tmp/tmpuu_9e2cq: line 1: export: `.': not a valid identifier
/tmp/tmpuu_9e2cq: line 1: export: `/rd22/scratch/broot/Programs/example/.tox/py38/etc/conda/activate.d/geotiff-activate.sh': not a valid identifier
/tmp/tmpuu_9e2cq: line 1: export: `.': not a valid identifier
/tmp/tmpuu_9e2cq: line 1: export: `/rd22/scratch/broot/Programs/example/.tox/py38/etc/conda/activate.d/proj4-activate.sh': not a valid identifier
foobar
__________________________________________________ summary ___________________________________________________
py38: commands succeeded
congratulations :)
(Yes, it succeeded, only because the command was an "echo" command instead of a usual pytest. In my actual tests, a bunch of the fiona and gdal calls are failing because the package activation scripts weren't sourced, as noted in the error output above.
This is weird... how is stuff like this in my log?
click-plugins @ file:///Users/ktietz/demo/mc3/conda-bld/click-plugins_1630665883852/work,
I am on a linux system, not a mac.
In case it is relevant,
(base) bash-4.4$ conda --version
conda 4.11.0
conda info --all is relevant.
(base) bash-4.4$ conda info --all
active environment : base
active env location : /home/broot/scratch/miniconda3
shell level : 1
user config file : /home/broot/.condarc
populated config files : /home/broot/.condarc
conda version : 4.11.0
conda-build version : not installed
python version : 3.7.10.final.0
virtual packages : __linux=4.18.0=0
__glibc=2.28=0
__unix=0=0
__archspec=1=x86_64
base environment : /home/broot/scratch/miniconda3 (writable)
conda av data dir : /home/broot/scratch/miniconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/broot/scratch/miniconda3/pkgs
/home/broot/.conda/pkgs
envs directories : /home/broot/scratch/miniconda3/envs
/home/broot/.conda/envs
platform : linux-64
user-agent : conda/4.11.0 requests/2.26.0 CPython/3.7.10 Linux/4.18.0-348.2.1.el8_5.x86_64 centos/8.5.2111 glibc/2.28
UID:GID : [snipped]
netrc file : /home/broot/.netrc
offline mode : False
# conda environments:
#
base * /home/broot/scratch/miniconda3
... [snipped ] ...
/rd22/scratch/broot/Programs/example/.tox/py38
sys.version: 3.7.10 | packaged by conda-forge | (defa...
sys.prefix: /home/broot/scratch/miniconda3
sys.executable: /home/broot/scratch/miniconda3/bin/python
conda location: /home/broot/scratch/miniconda3/lib/python3.7/site-packages/conda
conda-build: None
conda-env: /home/broot/scratch/miniconda3/bin/conda-env
user site dirs:
CIO_TEST: <not set>
CONDA_DEFAULT_ENV: base
CONDA_EXE: /home/broot/scratch/miniconda3/bin/conda
CONDA_PREFIX: /home/broot/scratch/miniconda3
CONDA_PROMPT_MODIFIER: (base)
CONDA_PYTHON_EXE: /home/broot/scratch/miniconda3/bin/python
CONDA_ROOT: /home/broot/scratch/miniconda3
CONDA_SHLVL: 1
CURL_CA_BUNDLE: <not set>
PATH: /home/broot/scratch/miniconda3/bin:/home/broot/scratch/miniconda3/bin:/home/broot/scratch/miniconda3/condabin:[snipped]:/home/broot/.local/bin
PYTHONSTARTUP: /home/broot/.pythonrc
REQUESTS_CA_BUNDLE: <not set>
SSL_CERT_FILE: <not set>
Snipped out possibly sensitive information such as my other environments, user id, and parts of my PATH.
Thank you, I am able to reproduce it and a fix is coming.
Thank you for your prompt response on this!
Version 0.9.1 has just been released with the fix.