Giters

microsoft / nlp-recipes

Natural Language Processing Best Practices & Examples

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[BUG] Installation broken

MikeyBeez opened this issue · comments

commented

Description

Thank you for sharing this interesting software. I tried to do a clean install on Pop!_OS 20.04 using your automated tool. Pip failed because of dependency conflicts.

` g clone git@github.com:MikeyBeez/nlp-recipes.git
Cloning into 'nlp-recipes'...
remote: Enumerating objects: 14854, done.
remote: Counting objects: 100% (40/40), done.
remote: Compressing objects: 100% (27/27), done.
remote: Total 14854 (delta 17), reused 20 (delta 12), pack-reused 14814
Receiving objects: 100% (14854/14854), 47.60 MiB | 24.83 MiB/s, done.
Resolving deltas: 100% (10732/10732), done.
❯ cd nlp-recipes
python tools/generate_conda_file.py

Generated conda file: nlp_cpu.yaml

To create the conda environment:
$ conda env create -f nlp_cpu.yaml

To update the conda environment:
$ conda env update -f nlp_cpu.yaml

To register the conda environment in Jupyter:
$ conda activate nlp_cpu
$ python -m ipykernel install --user --name nlp_cpu --display-name "Python (nlp_cpu)"

❯
❯ cp ../transfer-learning-conv-ai/.envrc .
direnv: error /media/home/bard/Code/nlp-recipes/.envrc is blocked. Run direnv allow to approve its content
❯ vim .envrc
direnv: error /media/home/bard/Code/nlp-recipes/.envrc is blocked. Run direnv allow to approve its content
❯ direnv allow
direnv: loading /media/home/bard/Code/nlp-recipes/.envrc
Could not find conda environment: nlp_cpu
You can list all discoverable environments with conda info --envs.

direnv: export ~PATH
❯ conda env create -f nlp_cpu.yaml
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages
textwrap3-0.9.2 | 12 KB | ################################### | 100%
wrapt-1.12.1 | 49 KB | ################################### | 100%
pytorch-cpu-1.3.1 | 2 KB | ################################### | 100%
termcolor-1.1.0 | 9 KB | ################################### | 100%
pytest-6.2.3 | 420 KB | ################################### | 100%
protobuf-3.14.0 | 303 KB | ################################### | 100%
distributed-1.28.1 | 826 KB | ################################### | 100%
grpcio-1.36.1 | 1.9 MB | ################################### | 100%
h5py-2.10.0 | 901 KB | ################################### | 100%
coverage-5.5 | 258 KB | ################################### | 100%
pluggy-0.13.1 | 33 KB | ################################### | 100%
dask-1.2.2 | 11 KB | ################################### | 100%
markdown-3.3.4 | 127 KB | ################################### | 100%
opt_einsum-3.3.0 | 57 KB | ################################### | 100%
hdf5-1.10.6 | 3.7 MB | ################################### | 100%
gstreamer-1.14.0 | 3.1 MB | ################################### | 100%
matplotlib-3.3.4 | 26 KB | ################################### | 100%
python-3.6.8 | 30.1 MB | ################################### | 100%
locket-0.2.1 | 10 KB | ################################### | 100%
msgpack-python-1.0.2 | 82 KB | ################################### | 100%
tenacity-7.0.0 | 32 KB | ################################### | 100%
papermill-1.2.1 | 50 KB | ################################### | 100%
absl-py-0.12.0 | 171 KB | ################################### | 100%
tensorflow-estimator | 271 KB | ################################### | 100%
keras-applications-1 | 29 KB | ################################### | 100%
tensorflow-base-1.15 | 87.5 MB | ################################### | 100%
dask-core-1.2.2 | 516 KB | ################################### | 100%
backports.tempfile-1 | 11 KB | ################################### | 100%
ansiwrap-0.8.4 | 9 KB | ################################### | 100%
pyyaml-5.4.1 | 170 KB | ################################### | 100%
werkzeug-0.16.1 | 258 KB | ################################### | 100%
tensorflow-hub-0.7.0 | 66 KB | ################################### | 100%
gast-0.2.2 | 155 KB | ################################### | 100%
astor-0.8.1 | 47 KB | ################################### | 100%
cffi-1.14.0 | 223 KB | ################################### | 100%
pytorch-1.3.1 | 24.9 MB | ################################### | 100%
tensorflow-1.15.0 | 4 KB | ################################### | 100%
psutil-5.8.0 | 329 KB | ################################### | 100%
matplotlib-base-3.3. | 5.1 MB | ################################### | 100%
kiwisolver-1.3.1 | 80 KB | ################################### | 100%
bokeh-2.3.2 | 5.8 MB | ################################### | 100%
backports.weakref-1. | 8 KB | ################################### | 100%
tensorboard-1.15.0 | 3.2 MB | ################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: | Ran pip subprocess with arguments:
['/home/bard/miniconda3/envs/nlp_cpu/bin/python', '-m', 'pip', 'install', '-U', '-r', '/media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt']
Pip subprocess output:
Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.1.0/en_core_web_sm-2.1.0.tar.gz (from -r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 21))
Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.1.0/en_core_web_sm-2.1.0.tar.gz (11.1 MB)
Obtaining s2s-ft from git+https://github.com/microsoft/unilm.git@s2s-ft.v0.3#egg=s2s-ft&subdirectory=s2s-ft (from -r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 35))
Cloning https://github.com/microsoft/unilm.git (to revision s2s-ft.v0.3) to ./src/s2s-ft
Collecting allennlp==0.8.4
Downloading allennlp-0.8.4-py3-none-any.whl (5.7 MB)
Collecting azureml-sdk[automl,contrib,notebooks]==1.0.85
Downloading azureml_sdk-1.0.85-py3-none-any.whl (4.6 kB)
Collecting azureml-train-automl==1.0.85
Downloading azureml_train_automl-1.0.85-py3-none-any.whl (3.4 kB)
Collecting azureml-dataprep==1.1.8
Downloading azureml_dataprep-1.1.8-py3-none-any.whl (24.0 MB)
Collecting azureml-widgets==1.0.85
Downloading azureml_widgets-1.0.85-py3-none-any.whl (13.9 MB)
Collecting azureml-mlflow==1.0.85
Downloading azureml_mlflow-1.0.85-py2.py3-none-any.whl (21 kB)
Collecting black>=18.6b4
Downloading black-21.5b2-py3-none-any.whl (138 kB)
Collecting cached-property==1.5.1
Downloading cached_property-1.5.1-py2.py3-none-any.whl (6.0 kB)
Collecting jsonlines>=1.2.0
Downloading jsonlines-2.0.0-py3-none-any.whl (6.3 kB)
Collecting nteract-scrapbook>=0.2.1
Downloading nteract_scrapbook-0.4.2-py3-none-any.whl (34 kB)
Collecting pydocumentdb>=2.3.3
Downloading pydocumentdb-2.3.5-py3-none-any.whl (93 kB)
Collecting pytorch-pretrained-bert>=0.6
Downloading pytorch_pretrained_bert-0.6.2-py3-none-any.whl (123 kB)
Collecting tqdm==4.32.2
Downloading tqdm-4.32.2-py2.py3-none-any.whl (50 kB)
Collecting pyemd==0.5.1
Downloading pyemd-0.5.1.tar.gz (91 kB)
Collecting ipywebrtc==0.4.3
Downloading ipywebrtc-0.4.3.tar.gz (530 kB)
Collecting pre-commit>=1.14.4
Downloading pre_commit-2.13.0-py2.py3-none-any.whl (190 kB)
Collecting scikit-learn<=0.20.3,>=0.19.0
Downloading scikit_learn-0.20.3-cp36-cp36m-manylinux1_x86_64.whl (5.4 MB)
Collecting seaborn>=0.9.0
Downloading seaborn-0.11.1-py3-none-any.whl (285 kB)
Collecting sklearn-crfsuite>=0.3.6
Downloading sklearn_crfsuite-0.3.6-py2.py3-none-any.whl (12 kB)
Collecting spacy==2.1.8
Downloading spacy-2.1.8-cp36-cp36m-manylinux1_x86_64.whl (30.8 MB)
Collecting transformers==2.9.0
Downloading transformers-2.9.0-py3-none-any.whl (635 kB)
Collecting gensim>=3.7.0
Downloading gensim-4.0.1-cp36-cp36m-manylinux1_x86_64.whl (23.9 MB)
Collecting nltk>=3.4
Downloading nltk-3.6.2-py3-none-any.whl (1.5 MB)
Collecting seqeval>=0.0.12
Downloading seqeval-1.2.2.tar.gz (43 kB)
Collecting pyrouge>=0.1.3
Downloading pyrouge-0.1.3.tar.gz (60 kB)
Collecting py-rouge>=1.1
Downloading py_rouge-1.1-py3-none-any.whl (56 kB)
Collecting indic-nlp-library>=0.6
Downloading indic_nlp_library-0.81-py3-none-any.whl (40 kB)
Collecting torchtext>=0.4.0
Downloading torchtext-0.9.1-cp36-cp36m-manylinux1_x86_64.whl (7.1 MB)
Collecting multiprocess==0.70.9
Downloading multiprocess-0.70.9.tar.gz (1.6 MB)
Collecting tensorboardX==1.8
Downloading tensorboardX-1.8-py2.py3-none-any.whl (216 kB)
Requirement already satisfied: Cython>=0.29.13 in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from -r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 32)) (0.29.23)
Collecting googledrivedownloader>=0.4
Downloading googledrivedownloader-0.4-py2.py3-none-any.whl (3.9 kB)
Collecting methodtools
Downloading methodtools-0.4.3.tar.gz (3.8 kB)
Collecting requests==2.22.0
Downloading requests-2.22.0-py2.py3-none-any.whl (57 kB)
Collecting requests-oauthlib==1.2.0
Downloading requests_oauthlib-1.2.0-py2.py3-none-any.whl (22 kB)
Collecting regex==2020.2.20
Downloading regex-2020.2.20-cp36-cp36m-manylinux2010_x86_64.whl (690 kB)
Requirement already satisfied: numpy in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from s2s-ft->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 35)) (1.19.2)
Collecting boto3
Downloading boto3-1.17.88-py2.py3-none-any.whl (131 kB)
Collecting sentencepiece
Downloading sentencepiece-0.1.95-cp36-cp36m-manylinux2014_x86_64.whl (1.2 MB)
Collecting sacremoses
Downloading sacremoses-0.0.45-py3-none-any.whl (895 kB)
Collecting jsonnet>=0.10.0
Downloading jsonnet-0.17.0.tar.gz (259 kB)
Requirement already satisfied: scipy in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from allennlp==0.8.4->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 1)) (1.5.2)
Collecting awscli>=1.11.91
Downloading awscli-1.19.88-py2.py3-none-any.whl (3.6 MB)
Collecting numpydoc>=0.8.0
Downloading numpydoc-1.1.0-py3-none-any.whl (47 kB)
Requirement already satisfied: pytz>=2017.3 in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from allennlp==0.8.4->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 1)) (2021.1)
Requirement already satisfied: matplotlib>=2.2.3 in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from allennlp==0.8.4->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 1)) (3.3.4)
Requirement already satisfied: torch>=0.4.1 in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from allennlp==0.8.4->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 1)) (1.3.1)
Collecting overrides
Downloading overrides-6.1.0-py3-none-any.whl (14 kB)
Collecting editdistance
Downloading editdistance-0.5.3-cp36-cp36m-manylinux1_x86_64.whl (178 kB)
Collecting gevent>=1.3.6
Downloading gevent-21.1.2-cp36-cp36m-manylinux2010_x86_64.whl (5.5 MB)
Requirement already satisfied: pytest in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from allennlp==0.8.4->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 1)) (6.2.3)
Collecting jsonpickle
Downloading jsonpickle-2.0.0-py2.py3-none-any.whl (37 kB)
Collecting word2number>=1.1
Downloading word2number-1.1.zip (9.7 kB)
Collecting unidecode
Downloading Unidecode-1.2.0-py2.py3-none-any.whl (241 kB)
Collecting responses>=0.7
Downloading responses-0.13.3-py2.py3-none-any.whl (25 kB)
Collecting parsimonious>=0.8.0
Downloading parsimonious-0.8.1.tar.gz (45 kB)
Collecting flaky
Downloading flaky-3.7.0-py2.py3-none-any.whl (22 kB)
Collecting flask-cors>=3.0.7
Downloading Flask_Cors-3.0.10-py2.py3-none-any.whl (14 kB)
Collecting ftfy
Downloading ftfy-6.0.3.tar.gz (64 kB)
Collecting conllu==0.11
Downloading conllu-0.11-py2.py3-none-any.whl (6.8 kB)
Collecting sqlparse>=0.2.4
Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
Collecting flask>=1.0.2
Downloading Flask-2.0.1-py3-none-any.whl (94 kB)
Requirement already satisfied: h5py in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from allennlp==0.8.4->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 1)) (2.10.0)
Collecting azureml-dataprep[fuse,pandas]<1.2.0a,>=1.1.37a
Downloading azureml_dataprep-1.1.38-py3-none-any.whl (26.9 MB)
Collecting azureml-train-automl-client==1.0.85.*
Downloading azureml_train_automl_client-1.0.85.4-py3-none-any.whl (69 kB)
Collecting azureml-automl-runtime==1.0.85.*
Downloading azureml_automl_runtime-1.0.85.6-py3-none-any.whl (1.9 MB)
Collecting azureml-automl-core==1.0.85.*
Downloading azureml_automl_core-1.0.85.5-py3-none-any.whl (102 kB)
Collecting azureml-train-automl-runtime==1.0.85.*
Downloading azureml_train_automl_runtime-1.0.85.5-py3-none-any.whl (77 kB)
Collecting dotnetcore2==2.1.8
Downloading dotnetcore2-2.1.8-py3-none-manylinux1_x86_64.whl (29.3 MB)
Collecting azureml-dataprep-native<14.0.0,>=13.0.0
Downloading azureml_dataprep_native-13.2.0-cp36-cp36m-manylinux1_x86_64.whl (1.3 MB)
Collecting azureml-telemetry==1.0.85.*
Downloading azureml_telemetry-1.0.85.2-py3-none-any.whl (29 kB)
Collecting azureml-core==1.0.85.*
Downloading azureml_core-1.0.85.6-py2.py3-none-any.whl (1.2 MB)
Requirement already satisfied: ipywidgets>=7.0.0 in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from azureml-widgets==1.0.85->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 5)) (7.6.3)
Collecting mlflow>=1.0.0
Downloading mlflow-1.17.0-py3-none-any.whl (14.2 MB)
Collecting srsly<1.1.0,>=0.0.6
Downloading srsly-1.0.5-cp36-cp36m-manylinux2014_x86_64.whl (184 kB)
Collecting murmurhash<1.1.0,>=0.28.0
Downloading murmurhash-1.0.5-cp36-cp36m-manylinux2014_x86_64.whl (20 kB)
Collecting preshed<2.1.0,>=2.0.1
Downloading preshed-2.0.1-cp36-cp36m-manylinux1_x86_64.whl (83 kB)
Collecting blis<0.3.0,>=0.2.2
Downloading blis-0.2.4-cp36-cp36m-manylinux1_x86_64.whl (3.2 MB)
Collecting thinc<7.1.0,>=7.0.8
Downloading thinc-7.0.8-cp36-cp36m-manylinux1_x86_64.whl (2.1 MB)
Collecting plac<1.0.0,>=0.9.6
Downloading plac-0.9.6-py2.py3-none-any.whl (20 kB)
Collecting cymem<2.1.0,>=2.0.2
Downloading cymem-2.0.5-cp36-cp36m-manylinux2014_x86_64.whl (35 kB)
Collecting wasabi<1.1.0,>=0.2.0
Downloading wasabi-0.8.2-py3-none-any.whl (23 kB)
Collecting dataclasses
Downloading dataclasses-0.8-py3-none-any.whl (19 kB)
Collecting filelock
Downloading filelock-3.0.12-py3-none-any.whl (7.6 kB)
Collecting tokenizers==0.7.0
Downloading tokenizers-0.7.0-cp36-cp36m-manylinux1_x86_64.whl (3.8 MB)
Collecting dill>=0.3.1
Downloading dill-0.3.3-py2.py3-none-any.whl (81 kB)
Requirement already satisfied: six in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from tensorboardX==1.8->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 31)) (1.15.0)
Requirement already satisfied: protobuf>=3.2.0 in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from tensorboardX==1.8->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 31)) (3.14.0)
Collecting chardet<3.1.0,>=3.0.2
Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Requirement already satisfied: certifi>=2017.4.17 in /media/home/bard/miniconda3/envs/nlp_cpu/lib/python3.6/site-packages (from requests==2.22.0->-r /media/home/bard/Code/nlp-recipes/condaenv.il8jw3q0.requirements.txt (line 36)) (2021.5.30)
Collecting idna<2.9,>=2.5
Downloading idna-2.8-py2.py3-none-any.whl (58 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
Collecting oauthlib>=3.0.0
Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting azureml-automl-core==1.0.85.*
Downloading azureml_automl_core-1.0.85.4-py3-none-any.whl (102 kB)
Downloading azureml_automl_core-1.0.85.2-py3-none-any.whl (102 kB)
Downloading azureml_automl_core-1.0.85.1-py3-none-any.whl (102 kB)
Downloading azureml_automl_core-1.0.85-py3-none-any.whl (102 kB)
INFO: pip is looking at multiple versions of regex to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of requests-oauthlib to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of requests to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of tensorboardx to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of multiprocess to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of transformers to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of spacy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of ipywebrtc to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pyemd to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of tqdm to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cached-property to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of azureml-mlflow to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of azureml-widgets to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of azureml-dataprep to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of azureml-train-automl to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of allennlp to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of en-core-web-sm to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of s2s-ft to determine which version is compatible with other requirements. This could take a while.

The conflict is caused by:
The user requested azureml-dataprep==1.1.8
azureml-automl-core 1.0.85.5 depends on azureml-dataprep<1.2.0a and >=1.1.37a
The user requested azureml-dataprep==1.1.8
azureml-automl-core 1.0.85.4 depends on azureml-dataprep<1.2.0a and >=1.1.37a
The user requested azureml-dataprep==1.1.8
azureml-automl-core 1.0.85.2 depends on azureml-dataprep<1.2.0a and >=1.1.37a
The user requested azureml-dataprep==1.1.8
azureml-automl-core 1.0.85.1 depends on azureml-dataprep<1.2.0a and >=1.1.37a
The user requested azureml-dataprep==1.1.8
azureml-automl-core 1.0.85 depends on azureml-dataprep<1.2.0a and >=1.1.37a

To fix this you could try to:

  1. loosen the range of package versions you've specified
  2. remove package versions to allow pip attempt to solve the dependency conflict

Pip subprocess error:
Running command git clone -q https://github.com/microsoft/unilm.git /media/home/bard/Code/nlp-recipes/src/s2s-ft
Running command git checkout -q 13641268b59df5cf90d27b451d87ab58b6a07055
ERROR: Cannot install azureml-dataprep==1.1.8 and azureml-train-automl because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

failed

CondaEnvException: Pip failed

Other Comments