errors importing native C library (python 3.8.6 & swig 4.0.1)
tomcucinotta opened this issue · comments
Hi, I'm recompiling somoclu on Ubuntu 20.10, with python 3.8.6, and I'm having this problem
- ./autogen.sh
- ./configure
- make => the src/libsomoclu.so library is compiled, and the src/somoclu binary works
- make python; sudo make python_install => all seems fine (see below)
[...]
Installed /usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg
[...]
- but when attempting to use it, it fails to import the C native library
$ python
Python 3.8.6 (default, May 27 2021, 13:28:02)
[GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import somoclu
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/__init__.py", line 12, in <module>
from .train import Somoclu
File "/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/train.py", line 25, in <module>
from .somoclu_wrap import train as wrap_train
File "/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/somoclu_wrap.py", line 13, in <module>
from . import _somoclu_wrap
ImportError: cannot import name '_somoclu_wrap' from partially initialized module 'somoclu' (most likely due to a circular import) (/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/__init__.py)
the error is highlighted re-raise-ing the exception
Any clue ? AFAICR, I wasn't getting into this with python3.6 (if memory serves).
Thanks.
$ find /usr/local/lib/python3.8/dist-packages/ | grep somoclu
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/train.py
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/init.py
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/pycache
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/pycache/init.cpython-38.pyc
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/pycache/train.cpython-38.pyc
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/pycache/somoclu_wrap.cpython-38.pyc
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/somoclu/somoclu_wrap.py
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/_somoclu_wrap.cpython-38-x86_64-linux-gnu.so
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/_somoclu_wrap.py
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO/not-zip-safe
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO/PKG-INFO
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO/native_libs.txt
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO/top_level.txt
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO/dependency_links.txt
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO/requires.txt
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/EGG-INFO/SOURCES.txt
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/pycache
/usr/local/lib/python3.8/dist-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/pycache/_somoclu_wrap.cpython-38.pyc
tracked down to swig (4.0.1-5build1) generating a somoclu_wrap.py with these lines
# Import the low-level C/C++ module
if __package__ or "." in __name__:
from . import _somoclu_wrap
else:
import _somoclu_wrap
if I get rid of the "from ." and just do "import __somoclu_wrap", then I can import somoclu from python.
But this is not an area I'm familiar with, at all!
Ubuntu 20.10 has a means for downgrading swig:
sudo apt-get purge swig*
sudo apt-get install swig3.0
# this is needed otherwise make python won't find swig
sudo ln -s /usr/bin/swig3.0 /usr/bin/swig
then, after recompiling (make python) and installing (sudo make python_install), it imports correctly.
The code to find the native C library in the same auto-generated somoclu_wrap.py is now a hell of code:
from sys import version_info as _swig_python_version_info
if _swig_python_version_info >= (2, 7, 0):
def swig_import_helper():
import importlib
pkg = __name__.rpartition('.')[0]
mname = '.'.join((pkg, '_somoclu_wrap')).lstrip('.')
try:
return importlib.import_module(mname)
except ImportError:
return importlib.import_module('_somoclu_wrap')
_somoclu_wrap = swig_import_helper()
del swig_import_helper
elif _swig_python_version_info >= (2, 6, 0):
def swig_import_helper():
from os.path import dirname
import imp
fp = None
try:
fp, pathname, description = imp.find_module('_somoclu_wrap', [dirname(__file__)])
except ImportError:
import _somoclu_wrap
return _somoclu_wrap
try:
_mod = imp.load_module('_somoclu_wrap', fp, pathname, description)
finally:
if fp is not None:
fp.close()
return _mod
_somoclu_wrap = swig_import_helper()
del swig_import_helper
else:
import _somoclu_wrap
Any clue as to how to get this working on the latest swig and avoid this nightmare on recent distros ?
Thanks!
I tested in a anaconda python 3.8 env with swig 4 installed from anaconda in ubuntu 20.04. After building and installing. cd /home/user/anaconda3/envs/p38/lib/python3.8/site-packages/somoclu-1.7.5-py3.8-linux-x86_64.egg/ and cp _somoclu_wrap.cpython-38-x86_64-linux-gnu.so _somoclu_wrap.py somoclu seems to work (only when cd in this directory).
From digging some swig github issues I found this seems to work. You can test.