SapienzaNLP / ewiser

A Word Sense Disambiguation system integrating implicit and explicit external knowledge.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Installation issue

luigi-asprino opened this issue · comments

Hi,

I had some problems with

pip install torch-scatter==latest+${CUDA} torch-sparse==latest+${CUDA} -f https://pytorch-geometric.com/whl/torch-1.5.0.html

since "latest" tag seems to be no longer supported cf. this issue.
You can cope with this issue with the following commands in which the torch version is explicitly asserted

pip install torch-sparse -f https://pytorch-geometric.com/whl/torch-1.8.0+cu101.html
pip install torch-scatter -f https://pytorch-geometric.com/whl/torch-1.8.0+cu101.html

Thanks for the heads up, Luigi! I have updated README.md.

Quick report to help people out there (this gave me some headache...):

  • don't go yet to pytorch 1.9 yet (no wheels available, it will be even more adventurous :-))
  • use python 3.8 (I successfully used 3.8.8)
  • you can use the coda toolkit 10.2 wheels for sparse and scatter safely with python 3.8.8 (provided your env for coda toolkit 10.2...)

Since I took the opportunity to wipe out my old Ubuntu to do a fresh install, here are some installation notes that worked for me, I hope this will help someone in the future.

A fresh Ubuntu 20.04 server with openssh.

Install nvidia drivers  (this excludes X server things and include CUDA as default)

sudo mkdir /usr/lib/nvidia
sudo apt-get install --no-install-recommends nvidia-driver-450
# Reboot. Check that GPUs are visible using the command: nvidia-smi

Install conda

cd /tmp
wget -P /tmp https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh
bash /tmp/Anaconda3-2020.02-Linux-x86_64.sh
source ~/.bashrc
conda update --all

Install pytorch 1.8.0 and conda-env with cudatoolkit 10.2 , torch-scatter/sparse stuff in a python 3.8.8 env (careful to stick to these versions or you'll deal with torch-scatter issues : installing from some other wheels do not generate *_cuda.so libs, and using python 1.9 for which no wheels are available yet will result in source compilation).

pip uninstall torch-sparse torch-scatter #run mutliple times to be no local nor global version exists
conda create -n ewiser python=3.8.8 pip;
conda activate ewiser 
conda install pytorch=1.8.0 cudatoolkit=10.2 -c pytorch

pip install --no-index --no-cache-dir torch-scatter -f https://pytorch-geometric.com/whl/torch-1.8.0+cu102.html
pip install --no-index --no-cache-dir torch-scatter -f https://pytorch-geometric.com/whl/torch-1.8.0+cu102.html

/!\ Check that some *_cuda.so files are created in both:

  • /home/cgravier/anaconda3/envs/ewiser/lib/python3.8/site-packages/torch_sparse/
  • /home/cgravier/anaconda3/envs/ewiser/lib/python3.8/site-packages/torch_scatter/

Install ewiser prerequisites

sudo apt-get install g++
sudo apt-get install libtool autoconf build-essential pkg-config automake tcsh
pip install -U pip setuptools wheel
pip install -U spacy
python -m spacy download en

And wordnet for nltk:

$ python

>>> import nltk
>>> dler = nltk.downloader.Downloader()
>>> dler._update_index()
>>> dler.download('all') # or use dler.download('wordnet') only 

Install ewiser itself

git clone https://github.com/SapienzaNLP/ewiser.git
cd ewiser
pip install -r requirements.txt
pip install -e .

Infer WSD from command line:

python ewiser/spacy/disambiguate.py ~/Coding/research/ewiser.semcor+wngt.pt

To make this last step work, you need to take into account the following PR : #8 (manually or automatically once included in the repo)