ImportError [...]/lib/python3.10/site-packages/gluonnlp/data/fast_bert_tokenizer.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send
MBrede opened this issue · comments
I hope this is not an incredibly dumb issue, but googling did not produce any useable results.
I installed gluonnlp 0.10.0 using pip on a Ubuntu 22.04.1 LTS server under Python 3.10.6.
When I try to import gluon, I get the following error:
Traceback (most recent call last):
File "[...]/lib/python3.10/site-packages/IPython/core/interactiveshell.py", line 3460, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-4-e68c3b59001c>", line 1, in <module>
import gluonnlp as nlp
File "[...]/lib/python3.10/site-packages/gluonnlp/__init__.py", line 25, in <module>
from . import data
File "[...]/lib/python3.10/site-packages/gluonnlp/data/__init__.py", line 23, in <module>
from . import (batchify, candidate_sampler, conll, corpora, dataloader,
File "[...]/lib/python3.10/site-packages/gluonnlp/data/corpora/__init__.py", line 21, in <module>
from . import (google_billion_word, large_text_compression_benchmark, wikitext)
File "[...]/lib/python3.10/site-packages/gluonnlp/data/corpora/google_billion_word.py", line 34, in <module>
from ...vocab import Vocab
File "[...]/lib/python3.10/site-packages/gluonnlp/vocab/__init__.py", line 21, in <module>
from . import bert, elmo, subwords, vocab
File "[...]/lib/python3.10/site-packages/gluonnlp/vocab/bert.py", line 24, in <module>
from ..data.transforms import SentencepieceTokenizer
File "[...]/lib/python3.10/site-packages/gluonnlp/data/transforms.py", line 48, in <module>
from .fast_bert_tokenizer import is_control, is_punctuation, is_whitespace
ImportError: [...]/lib/python3.10/site-packages/gluonnlp/data/fast_bert_tokenizer.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send
pip list returns the following modules, if this information is helpfull:
Package Version
------------------------ -----------
asttokens 2.2.1
backcall 0.2.0
certifi 2022.12.7
charset-normalizer 3.0.1
click 8.1.3
contourpy 1.0.7
cvxpy 1.3.0
cycler 0.11.0
Cython 0.29.33
decorator 5.1.1
ecos 2.0.12
executing 1.2.0
fasttext 0.9.2
filelock 3.9.0
fonttools 4.38.0
fst-pso 1.8.1
FuzzyTM 2.0.5
gensim 4.3.0
germansentiment 1.1.0
gluonnlp 0.10.0
graphviz 0.8.4
h5py 3.8.0
hdbscan 0.8.29
huggingface-hub 0.12.1
idna 3.4
ipython 8.10.0
jedi 0.18.2
joblib 1.2.0
kaleido 0.2.1
kiwisolver 1.4.4
llvmlite 0.39.1
matplotlib 3.7.0
matplotlib-inline 0.1.6
miniful 0.0.6
mxnet 1.9.1
nltk 3.8.1
numba 0.56.4
numpy 1.23.5
nvidia-cublas-cu11 11.10.3.66
nvidia-cuda-nvrtc-cu11 11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11 8.5.0.96
osqp 0.6.2.post8
packaging 23.0
pandas 1.5.3
parso 0.8.3
pexpect 4.8.0
pickleshare 0.7.5
Pillow 9.4.0
pip 23.0.1
plotly 5.13.0
polars 0.16.7
prompt-toolkit 3.0.36
ptyprocess 0.7.0
pure-eval 0.2.2
pybind11 2.10.3
pyFUME 0.2.25
Pygments 2.14.0
pynndescent 0.5.8
pyparsing 3.0.9
pyportfolioopt 1.5.4
python-dateutil 2.8.2
pytz 2022.7.1
PyYAML 6.0
qdldl 0.1.5.post3
regex 2022.10.31
requests 2.28.2
scikit-learn 1.2.1
scipy 1.10.1
scs 3.2.2
sentence-transformers 2.2.2
sentencepiece 0.1.97
setuptools 64.0.2
simpful 2.9.0
simple-elmo 0.9.1
six 1.16.0
sklearn 0.0.post1
smart-open 6.3.0
Snowball 0.5.2
stack-data 0.6.2
tenacity 8.2.1
threadpoolctl 3.1.0
tokenizers 0.13.2
top2vec 1.0.28
torch 1.13.1
torchvision 0.14.1
tqdm 4.64.1
traitlets 5.9.0
transformers 4.26.1
typing 3.6.6
typing_extensions 4.5.0
umap-learn 0.5.3
urllib3 1.26.14
wcwidth 0.2.6
wheel 0.38.4
wordcloud 1.8.2.2
I tried reinstalling cython, gluonnlp and mxnet and would like to avoid using another python version if possible.
I am not sure whether this is a gluon or a Cython issue, I hope this is the right place to ask!
I did also post a question on StackOverflow regarding this issue.
It was the old mxnet version, sorry for any inconvenience.
@MBrede I'm running into this error as well. What version of mxnet are you using that fixed it?
I've tried installing 1.9.1 but it didn't appear to resolve it in my env at least.
@djherbis I installed the version mentioned in the README (python3 -m pip install -U --pre "mxnet>=2.0.0a") which did fix the aforementioned error but did end up in a new error (which I unfortunately do not remember). In the end, I wrote a module wrapping the models I wanted to use from other modules...
Thanks, yeah I was unable to get it to work with that command either :(
For now I've pulled it from the Kaggle image for low use anyway.
Thanks for responding though!
Downgrade python version