Error after installing on Ubuntu 18 on Windows Linux Subsystem
Munfred opened this issue · comments
Hello, I wanted to try scelvis out so I installed it using pip3 install scelvis on Ubuntu 18.04 with the Windows Linux Subsystem. I tried running it with scelvis and got the error below, apparently due to the botocore dependency. This was with Python 3.6.7.
Then I tried installing it on a linux machine with CentOS 7 and Python 3.7 using pip3 install scelvis --user and it worked fine. (but I couldn't access a port)
So I tried installing it on a small Google cloud instance with Ubuntu 18.04 and Python 3.6.8 and it worked, but the instance probably had too little ram because the process kept getting killed when I tried starting it on the demo dataset.
So I booted a bigger instance also with Ubuntu 18.04 and Python 3.6.8, but this time at the end of the installation I got (more than once) the error Segmentation fault (core dumped).
It's getting late so I'll try again tomorrow on another machine, but I just wanted to point these oddities out in case you might have an idea what causes them. Thanks
munfred@DESKTOP-R4LKK64:~$ scelvis
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 574, in _build_master
ws.require(__requires__)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 892, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 783, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (python-dateutil 2.8.1 (/home/munfred/.local/lib/python3.6/site-packages), Requirement.parse('python-dateutil<2.8.1,>=2.1; python_version >= "2.7"'), {'botocore'})
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/scelvis", line 6, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3088, in <module>
@_call_aside
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3072, in _call_aside
f(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 3101, in _initialize_master_working_set
working_set = WorkingSet._build_master()
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 576, in _build_master
return cls._build_from_requirements(__requires__)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 589, in _build_from_requirements
dists = ws.resolve(reqs, Environment())
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 783, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (python-dateutil 2.8.1 (/home/munfred/.local/lib/python3.6/site-packages), Requirement.parse('python-dateutil<2.8.1,>=2.1; python_version >= "2.7"'), {'botocore'})
Hi Munfred,
have you tried installing via conda or using the Docker image?
I got a similar error when doing pip install scelvis in python 3.6.8 and python 3.7 (within conda envs)
ERROR: botocore 1.13.16 has requirement python-dateutil<2.8.1,>=2.1; python_version >= "2.7", but you'll have python-dateutil 2.8.1 which is incompatible.
but scelvis run works fine for me nevertheless.
what do you mean by "couldn't access a port" - is that a problem with scelvis or with this particular machine?
thanks for your input!
That was an issue with the cluster, not with scelvis
I haven't tried conda install or using the docker image, is that the recommended approach?
Hi @Munfred, your best bet probably is installing Miniconda3 first (works fine for me in WSL) and then installing scelvis using conda install -y scelvis or conda create -n scelvis-env -y scelvis && conda activate scelvis-env && scelvis --help. For cloud usage, I'd recommend running in Docker.
pip is nice from an early 2000s perspective but the worst thing about Python is its packaging. We're looking into using pipenv to resolve such issues. We're using conda for Python package environment management here with good success but your mileage might vary.
Good luck, we're looking forward to learning about your experience!
Okay, so on WSL I was able to install with conda install -c bioconda scelvis (perhaps you can also mention in the readme you need the bioconda channel). Worked with no problems.
Then I tried installing on another PC with a fresh install of WSL using Ubuntu 18.04 and anaconda, and ran into the error below. I then realized that maybe it was just that I needed to update Ubuntu, so I did apt get update and apt get upgrade, but still got the same error when installing with conda. Not a big deal, but something to keep in mind down the road as more people try to install the package. I copy the two errors below in case it is relevant.
Error before upgrade:
(base) munfred@DESKTOP-PCTQLQT:/mnt/c/Users/munfred$ conda install -c bioconda scelvis
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata so
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Package zlib conflicts for:
python=3.7 -> zlib[version='>=1.2.11,<1.3.0a0']
Package dash-core-components conflicts for:
scelvis -> dash-core-components[version='<1.0']
Package flask conflicts for:
scelvis -> flask
Package readline conflicts for:
python=3.7 -> readline[version='>=7.0,<8.0a0']
Package python-irodsclient conflicts for:
scelvis -> python-irodsclient
Package dash-bootstrap-components conflicts for:
scelvis -> dash-bootstrap-components
Package pip conflicts for:
python=3.7 -> pip
Package tk conflicts for:
python=3.7 -> tk[version='>=8.6.7,<8.7.0a0|>=8.6.8,<8.7.0a0']
Package numpy conflicts for:
scelvis -> numpy
Package attrs conflicts for:
scelvis -> attrs
Package libstdcxx-ng conflicts for:
python=3.7 -> libstdcxx-ng[version='>=7.2.0|>=7.3.0']
Package ruamel_yaml conflicts for:
scelvis -> ruamel_yaml
Package dash-table conflicts for:
scelvis -> dash-table=3.7
Package fs.sshfs conflicts for:
scelvis -> fs.sshfs
Package flask-caching conflicts for:
scelvis -> flask-caching
Package openssl conflicts for:
python=3.7 -> openssl[version='>=1.0.2o,<1.0.3a|>=1.0.2p,<1.0.3a|>=1.1.1a,<1.1.2a|>=1.1.1b,<1.1.2a|>=1.1.1d,<1.1.2a']
Package dash-renderer conflicts for:
scelvis -> dash-renderer[version='<1.0']
Package dash conflicts for:
scelvis -> dash[version='<1.0']
Package plotly conflicts for:
scelvis -> plotly
Package anndata conflicts for:
scelvis -> anndata
Package libffi conflicts for:
python=3.7 -> libffi[version='>=3.2.1,<4.0a0']
Package htmllistparse conflicts for:
scelvis -> htmllistparse
Package s3fs conflicts for:
scelvis -> s3fs
Package fs conflicts for:
scelvis -> fs
Package ruamel.yaml conflicts for:
scelvis -> ruamel.yaml
Package xz conflicts for:
python=3.7 -> xz[version='>=5.2.4,<6.0a0']
Package libgcc-ng conflicts for:
python=3.7 -> libgcc-ng[version='>=7.2.0|>=7.3.0']
Package sqlite conflicts for:
python=3.7 -> sqlite[version='>=3.24.0,<4.0a0|>=3.25.2,<4.0a0|>=3.25.3,<4.0a0|>=3.26.0,<4.0a0|>=3.27.2,<4.0a0|>=3.29.0,<4.0a0|>=3.30.1,<4.0a0']
Package dash-html-components conflicts for:
scelvis -> dash-html-components[version='<1.0']
Package logzero conflicts for:
scelvis -> logzero
Package requests conflicts for:
scelvis -> requests
Package scanpy conflicts for:
scelvis -> scanpy
Package pandas conflicts for:
scelvis -> pandas
Package ncurses conflicts for:
python=3.7 -> ncurses[version='>=6.1,<7.0a0']
Error after upgrade:
(base) munfred@DESKTOP-PCTQLQT:/mnt/c/Users/munfred$ conda install -c bioconda scelvis
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Package ruamel.yaml conflicts for:
scelvis -> ruamel.yaml
Package htmllistparse conflicts for:
scelvis -> htmllistparse
Package libgcc-ng conflicts for:
python=3.7 -> libgcc-ng[version='>=7.2.0|>=7.3.0']
Package dash-bootstrap-components conflicts for:
scelvis -> dash-bootstrap-components
Package pip conflicts for:
python=3.7 -> pip
Package dash conflicts for:
scelvis -> dash[version='<1.0']
Package xz conflicts for:
python=3.7 -> xz[version='>=5.2.4,<6.0a0']
Package dash-core-components conflicts for:
scelvis -> dash-core-components[version='<1.0']
Package sqlite conflicts for:
python=3.7 -> sqlite[version='>=3.24.0,<4.0a0|>=3.25.2,<4.0a0|>=3.25.3,<4.0a0|>=3.26.0,<4.0a0|>=3.27.2,<4.0a0|>=3.29.0,<4.0a0|>=3.30.1,<4.0a0']
Package dash-renderer conflicts for:
scelvis -> dash-renderer[version='<1.0']
Package plotly conflicts for:
scelvis -> plotly
Package dash-html-components conflicts for:
scelvis -> dash-html-components[version='<1.0']
Package fs conflicts for:
scelvis -> fs
Package libffi conflicts for:
python=3.7 -> libffi[version='>=3.2.1,<4.0a0']
Package pandas conflicts for:
scelvis -> pandas
Package libstdcxx-ng conflicts for:
python=3.7 -> libstdcxx-ng[version='>=7.2.0|>=7.3.0']
Package ruamel_yaml conflicts for:
scelvis -> ruamel_yaml
Package ncurses conflicts for:
python=3.7 -> ncurses[version='>=6.1,<7.0a0']
Package logzero conflicts for:
scelvis -> logzero
Package scanpy conflicts for:
scelvis -> scanpy
Package flask conflicts for:
scelvis -> flask
Package s3fs conflicts for:
scelvis -> s3fs
Package openssl conflicts for:
python=3.7 -> openssl[version='>=1.0.2o,<1.0.3a|>=1.0.2p,<1.0.3a|>=1.1.1a,<1.1.2a|>=1.1.1b,<1.1.2a|>=1.1.1c,<1.1.2a|>=1.1.1d,<1.1.2a']
Package numpy conflicts for:
scelvis -> numpy
Package attrs conflicts for:
scelvis -> attrs
Package requests conflicts for:
scelvis -> requests
Package tk conflicts for:
python=3.7 -> tk[version='>=8.6.7,<8.7.0a0|>=8.6.8,<8.7.0a0']
Package python-irodsclient conflicts for:
scelvis -> python-irodsclient
Package readline conflicts for:
python=3.7 -> readline[version='>=7.0,<8.0a0']
Package fs.sshfs conflicts for:
scelvis -> fs.sshfs
Package dash-table conflicts for:
scelvis -> dash-table=3.7
Package flask-caching conflicts for:
scelvis -> flask-caching
Package zlib conflicts for:
python=3.7 -> zlib[version='>=1.2.11,<1.3.0a0']
Package anndata conflicts for:
scelvis -> anndata
Hi, interesting. The main issue here are conflicts that are upstream and out of our control. Maybe we should provide environment.yaml files with each release that are known to work.
How does your ~/.condarc look like?
Mine:
channels:
- conda-forge
- bioconda
- defaults
sat_solver: pycryptosat
You could also try conda create -n scelvis -y scelvis and see whether this works.
A clean reinstall of conda works for me:
# wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
# bash Miniconda3-latest-Linux-x86_64.sh -b -p /tmp/miniconda3
# source /tmp/miniconda3/bin/activate
# conda create -n scelvis -y scelvis
# conda activate scelvis
# scelvis --help
Here is an `environment.yaml` file for the created fresh environment.
name: scelvis
channels:
- conda-forge
- bioconda
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- anndata=0.6.22.post1=py_1
- appdirs=1.4.3=py_1
- attrs=19.3.0=py_0
- backports=1.0=py_2
- bcrypt=3.1.7=py37h516909a_0
- beautifulsoup4=4.8.1=py37_0
- blosc=1.17.0=he1b5a44_1
- boto3=1.10.17=py_0
- botocore=1.13.17=py_0
- bzip2=1.0.8=h516909a_1
- ca-certificates=2019.9.11=hecc5488_0
- cached-property=1.5.1=py_0
- cairo=1.16.0=hfb77d84_1002
- certifi=2019.9.11=py37_0
- cffi=1.13.2=py37h8022711_0
- chardet=3.0.4=py37_1003
- click=7.0=py_0
- configparser=3.7.3=py37_1
- contextlib2=0.6.0.post1=py_0
- cryptography=2.8=py37h72c5cf5_0
- cycler=0.10.0=py_2
- dash=1.4.1=py_0
- dash-bootstrap-components=0.7.2=py37_0
- dash-core-components=1.3.1=py_0
- dash-html-components=1.0.1=py_0
- dash-renderer=1.1.2=py_0
- dash-table=4.4.1=py_0
- dbus=1.13.6=he372182_0
- decorator=4.4.1=py_0
- docutils=0.15.2=py37_0
- expat=2.2.5=he1b5a44_1004
- flask=1.1.1=py_1
- flask-caching=1.7.1=py_0
- flask-compress=1.4.0=py_0
- fontconfig=2.13.1=h86ecdb6_1001
- freetype=2.10.0=he983fc9_1
- fs=2.3.0=py37_0
- fs.sshfs=0.10.3=py37_0
- fsspec=0.6.0=py_0
- fusepy=3.0.1=py_0
- future=0.18.2=py37_0
- gettext=0.19.8.1=hc5be6a0_1002
- glib=2.58.3=py37h6f030ca_1002
- gmp=6.1.2=hf484d3e_1000
- gst-plugins-base=1.14.5=h0935bb2_0
- gstreamer=1.14.5=h36ae1b5_0
- h5py=2.10.0=nompi_py37h513d04c_100
- hdf5=1.10.5=nompi_h3c11f04_1104
- html5lib=1.0.1=py_0
- htmllistparse=0.5=py_0
- icu=64.2=he1b5a44_1
- idna=2.8=py37_1000
- igraph=0.7.1=h9e3b1fc_1007
- importlib-metadata=0.19=py37_0
- itsdangerous=1.1.0=py_0
- jinja2=2.10.3=py_0
- jmespath=0.9.4=py_0
- joblib=0.14.0=py_0
- jpeg=9c=h14c3975_1001
- kiwisolver=1.1.0=py37hc9558a2_0
- libblas=3.8.0=14_openblas
- libcblas=3.8.0=14_openblas
- libclang=9.0.0=default_hde54327_4
- libffi=3.2.1=he1b5a44_1006
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_2
- libiconv=1.15=h516909a_1005
- liblapack=3.8.0=14_openblas
- libllvm8=8.0.1=hc9558a2_0
- libllvm9=9.0.0=hc9558a2_3
- libopenblas=0.3.7=h6e990d7_3
- libpng=1.6.37=hed695b0_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libuuid=2.32.1=h14c3975_1000
- libxcb=1.13=h14c3975_1002
- libxkbcommon=0.9.1=hebb1f50_0
- libxml2=2.9.10=hee79883_0
- llvmlite=0.30.0=py37h8b12597_1
- logzero=1.5.0=py37_1001
- louvain=0.6.1=py37he1b5a44_2
- lzo=2.10=h14c3975_1000
- markupsafe=1.1.1=py37h516909a_0
- matplotlib=3.1.2=py37_1
- matplotlib-base=3.1.2=py37h250f245_1
- mock=3.0.5=py37_0
- more-itertools=7.2.0=py_0
- natsort=6.2.0=py_0
- ncurses=6.1=hf484d3e_1002
- networkx=2.4=py_0
- nspr=4.23=he1b5a44_0
- nss=3.47=he751ad9_0
- numba=0.46.0=py37hb3f55d8_1
- numexpr=2.7.0=py37hb3f55d8_0
- numpy=1.17.3=py37h95a1406_0
- openssl=1.1.1d=h516909a_0
- pandas=0.25.3=py37hb3f55d8_0
- paramiko=2.6.0=py37_0
- pathlib2=2.3.5=py37_0
- patsy=0.5.1=py_0
- pcre=8.43=he1b5a44_0
- pip=19.3.1=py37_0
- pixman=0.38.0=h516909a_1003
- plotly=4.3.0=py_0
- prettytable=0.7.2=py_3
- pthread-stubs=0.4=h14c3975_1001
- pycairo=1.18.2=py37h438ddbb_0
- pycparser=2.19=py37_1
- pynacl=1.3.0=py37h516909a_1001
- pyopenssl=19.0.0=py37_0
- pyparsing=2.4.5=py_0
- pyqt=5.12.3=py37hcca6a23_1
- pysocks=1.7.1=py37_0
- pytables=3.6.1=py37h9f153d1_0
- python=3.7.3=h33d41f4_1
- python-dateutil=2.8.0=py_0
- python-igraph=0.7.1.post7=py37h516909a_0
- python-irodsclient=0.7.0=py_0
- pytz=2019.3=py_0
- pyyaml=5.1.2=py37h516909a_0
- qt=5.12.5=hd8c4c69_1
- readline=8.0=hf8c457e_0
- requests=2.22.0=py37_1
- retrying=1.3.3=py_2
- ruamel.yaml=0.16.5=py37h516909a_1
- ruamel.yaml.clib=0.2.0=py37h516909a_0
- s3fs=0.4.0=py_0
- s3transfer=0.2.1=py37_0
- scanpy=1.4.4.post1=py_0
- scelvis=0.7.2=py_0
- scikit-learn=0.21.3=py37hcdab131_0
- scipy=1.3.2=py37h921218d_0
- seaborn=0.9.0=py_2
- setuptools=41.6.0=py37_1
- six=1.13.0=py37_0
- soupsieve=1.9.4=py37_0
- sqlite=3.30.1=hcee41ef_0
- statsmodels=0.10.1=py37hc1659b7_2
- tk=8.6.9=hed695b0_1003
- tornado=6.0.3=py37h516909a_0
- tqdm=4.38.0=py_0
- umap-learn=0.3.10=py37_0
- urllib3=1.25.7=py37_0
- webencodings=0.5.1=py_1
- werkzeug=0.16.0=py_0
- wheel=0.33.6=py37_0
- xmlrunner=1.7.7=py_0
- xorg-kbproto=1.0.7=h14c3975_1002
- xorg-libice=1.0.10=h516909a_0
- xorg-libsm=1.2.3=h84519dc_1000
- xorg-libx11=1.6.9=h516909a_0
- xorg-libxau=1.0.9=h14c3975_0
- xorg-libxdmcp=1.1.3=h516909a_0
- xorg-libxext=1.3.4=h516909a_0
- xorg-libxrender=0.9.10=h516909a_1002
- xorg-renderproto=0.11.1=h14c3975_1002
- xorg-xextproto=7.3.0=h14c3975_1002
- xorg-xproto=7.0.31=h14c3975_1007
- xz=5.2.4=h14c3975_1001
- yaml=0.1.7=h14c3975_1001
- zipp=0.6.0=py_0
- zlib=1.2.11=h516909a_1006
- pip:
- pyqt5-sip==4.19.18
- pyqtwebengine==5.12.1
PS: for long text, you could look into collapsible Github Markdown.