symbol not found
jingpengw opened this issue · comments
it is probably some library version conflict in my environment.
When I installed z5py, libboost 1.77 was installed.
has anyone seen this error and has a solution?
import z5py
File "/Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/__init__.py", line 1, in <module>
from .file import File, N5File, ZarrFile, S3File
File "/Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/file.py", line 6, in <module>
from . import _z5py
ImportError: dlopen(/Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so, 2): Symbol not found: __ZN5boost10filesystem4path25m_path_iterat
or_incrementERNS1_8iteratorE
Referenced from: /Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so
Expected in: /Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/../../../libboost_filesystem.dylib
in /Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so
Would suggest including enough info to explain how things were installed so someone else can reproduce. If Conda, conda list along with commands and channel used. Similar info for other install mechanisms
this is the output of conda info:
conda info
active environment : cf
active env location : /Users/jwu/opt/anaconda3/envs/cf
shell level : 2
user config file : /Users/jwu/.condarc
populated config files : /Users/jwu/.condarc
conda version : 4.10.3
conda-build version : 3.18.11
python version : 3.8.3.final.0
virtual packages : __osx=10.16=0
__unix=0=0
__archspec=1=x86_64
base environment : /Users/jwu/opt/anaconda3 (writable)
conda av data dir : /Users/jwu/opt/anaconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/osx-64
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/osx-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /Users/jwu/opt/anaconda3/pkgs
/Users/jwu/.conda/pkgs
envs directories : /Users/jwu/opt/anaconda3/envs
/Users/jwu/.conda/envs
platform : osx-64
user-agent : conda/4.10.3 requests/2.25.1 CPython/3.8.3 Darwin/20.6.0 OSX/10.16
UID:GID : 1647:20
netrc file : None
offline mode : False
I removed the whole conda environment and reinstalled z5py, still got the same error. I used python 3.8 this time. These are the packages I got.
Package Version Build Channel Size
───────────────────────────────────────────────────────────────────────────────
Install:
───────────────────────────────────────────────────────────────────────────────
+ blosc 1.21.0 he49afe7_0 conda-forge/osx-64 920 KB
+ boost-cpp 1.77.0 hff03dee_0 conda-forge/osx-64 16 MB
+ libblas 3.9.0 12_osx64_openblas conda-forge/osx-64 12 KB
+ libcblas 3.9.0 12_osx64_openblas conda-forge/osx-64 12 KB
+ libgfortran 5.0.0 9_3_0_h6c81a4c_23 conda-forge/osx-64 19 KB
+ libgfortran5 9.3.0 h6c81a4c_23 conda-forge/osx-64 2 MB
+ liblapack 3.9.0 12_osx64_openblas conda-forge/osx-64 12 KB
+ libopenblas 0.3.18 openmp_h3351f45_0 conda-forge/osx-64 9 MB
+ llvm-openmp 12.0.1 hda6cdc1_1 conda-forge/osx-64 287 KB
+ numpy 1.21.3 py38h49b9922_1 conda-forge/osx-64 6 MB
+ z5py 2.0.9 py38h2720e7c_0 conda-forge/osx-64 619 KB
Change:
───────────────────────────────────────────────────────────────────────────────
- certifi 2021.10.8 py38hecd8cb5_0 installed
+ certifi 2021.10.8 py38h50d1736_1 conda-forge/osx-64 145 KB
- openssl 1.1.1l h9ed2024_0 installed
+ openssl 1.1.1l h0d85af4_0 conda-forge/osx-64 2 MB
Summary:
Install: 11 packages
Change: 2 packages
z5py used to work in my laptop with python 3.7. I need to upgrade to 3.8 or 3.9 since another package used new features of python.
Also please include conda list
# packages in environment at /Users/jwu/opt/anaconda3/envs/cf:
#
# Name Version Build Channel
attrs 21.2.0 pypi_0 pypi
blosc 1.21.0 he49afe7_0 conda-forge
boost-cpp 1.77.0 hff03dee_0 conda-forge
boto3 1.19.10 pypi_0 pypi
botocore 1.22.11 pypi_0 pypi
brotli 1.0.9 pypi_0 pypi
brotlipy 0.7.0 py38h96a0964_1003 conda-forge
bzip2 1.0.8 h0d85af4_4 conda-forge
c-ares 1.18.1 h0d85af4_0 conda-forge
ca-certificates 2021.10.26 hecd8cb5_2
cachetools 4.2.4 pypi_0 pypi
certifi 2021.10.8 py38h50d1736_1 conda-forge
cffi 1.14.6 py38h2125817_0
chardet 4.0.0 py38h50d1736_1 conda-forge
charset-normalizer 2.0.0 pyhd8ed1ab_0 conda-forge
chunkflow 1.0.8 dev_0 <develop>
click 8.0.3 pypi_0 pypi
cloud-files 4.3.0 pypi_0 pypi
cloud-volume 5.3.2 pypi_0 pypi
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
compressed-segmentation 2.0.1 pypi_0 pypi
compresso 2.3.1 pypi_0 pypi
conda 4.10.3 py38h50d1736_3 conda-forge
conda-package-handling 1.7.3 py38h96a0964_1 conda-forge
connected-components-3d 3.3.0 pypi_0 pypi
crc32c 2.2.post0 pypi_0 pypi
cryptography 35.0.0 py38h56c4533_2 conda-forge
cycler 0.11.0 pypi_0 pypi
deflate 0.3.0 pypi_0 pypi
dijkstra3d 1.12.0 pypi_0 pypi
dill 0.3.4 pypi_0 pypi
docutils 0.18 pypi_0 pypi
dracopy 0.0.19 pypi_0 pypi
edt 2.1.1 pypi_0 pypi
fasteners 0.16.3 pypi_0 pypi
fastremap 1.12.2 pypi_0 pypi
fill-voids 2.0.2 pypi_0 pypi
fpzip 1.1.5 pypi_0 pypi
gevent 21.8.0 pypi_0 pypi
google-api-core 2.2.2 pypi_0 pypi
google-apitools 0.5.32 pypi_0 pypi
google-auth 2.3.3 pypi_0 pypi
google-cloud-core 2.1.0 pypi_0 pypi
google-cloud-storage 1.42.3 pypi_0 pypi
google-crc32c 1.3.0 pypi_0 pypi
google-resumable-media 2.1.0 pypi_0 pypi
googleapis-common-protos 1.53.0 pypi_0 pypi
greenlet 1.1.2 pypi_0 pypi
h5py 3.5.0 pypi_0 pypi
httplib2 0.20.2 pypi_0 pypi
icu 68.2 he49afe7_0 conda-forge
idna 3.1 pyhd3deb0d_0 conda-forge
imageio 2.10.1 pypi_0 pypi
inflection 0.5.1 pypi_0 pypi
iniconfig 1.1.1 pypi_0 pypi
jmespath 0.10.0 pypi_0 pypi
joblib 1.1.0 pypi_0 pypi
json5 0.9.6 pypi_0 pypi
jsonschema 3.2.0 pypi_0 pypi
kimimaro 3.0.0 pypi_0 pypi
kiwisolver 1.3.2 pypi_0 pypi
krb5 1.19.2 hcfbf3a7_3 conda-forge
libarchive 3.5.2 h2b60450_1 conda-forge
libblas 3.9.0 12_osx64_openblas conda-forge
libcblas 3.9.0 12_osx64_openblas conda-forge
libcurl 7.79.1 hf45b732_1 conda-forge
libcxx 12.0.0 h2f01273_0
libedit 3.1.20210910 hca72f7f_0
libev 4.33 haf1e3a3_1 conda-forge
libffi 3.3 hb1e8313_2
libgfortran 5.0.0 9_3_0_h6c81a4c_23 conda-forge
libgfortran5 9.3.0 h6c81a4c_23 conda-forge
libiconv 1.16 haf1e3a3_0 conda-forge
liblapack 3.9.0 12_osx64_openblas conda-forge
libnghttp2 1.43.0 h6f36284_1 conda-forge
libopenblas 0.3.18 openmp_h3351f45_0 conda-forge
libsolv 0.7.19 hcf210ce_5 conda-forge
libssh2 1.10.0 h52ee1ee_2 conda-forge
libxml2 2.9.12 h93ec3fd_0 conda-forge
llvm-openmp 12.0.1 hda6cdc1_1 conda-forge
lz4-c 1.9.3 he49afe7_1 conda-forge
lzo 2.10 haf1e3a3_1000 conda-forge
mamba 0.17.0 py38h0c98490_0 conda-forge
markdown 3.3.4 pypi_0 pypi
matplotlib 3.4.3 pypi_0 pypi
multiprocess 0.70.12.2 pypi_0 pypi
ncurses 6.3 hca72f7f_0
networkx 2.6.3 pypi_0 pypi
neuroglancer 2.22 pypi_0 pypi
numpy 1.21.3 py38h49b9922_1 conda-forge
oauth2client 4.1.3 pypi_0 pypi
openssl 1.1.1l h0d85af4_0 conda-forge
orjson 3.6.4 pypi_0 pypi
packaging 21.2 pypi_0 pypi
pandas 1.3.4 pypi_0 pypi
pathos 0.2.8 pypi_0 pypi
pillow 8.4.0 pypi_0 pypi
pip 21.2.4 py38hecd8cb5_0
pluggy 1.0.0 pypi_0 pypi
posix-ipc 1.0.5 pypi_0 pypi
pox 0.3.0 pypi_0 pypi
ppft 1.6.6.4 pypi_0 pypi
protobuf 3.19.1 pypi_0 pypi
psutil 5.8.0 pypi_0 pypi
py 1.11.0 pypi_0 pypi
pyasn1 0.4.8 pypi_0 pypi
pyasn1-modules 0.2.8 pypi_0 pypi
pycosat 0.6.3 py38h96a0964_1009 conda-forge
pycparser 2.20 pyh9f0ad1d_2 conda-forge
pynrrd 0.4.2 pypi_0 pypi
pyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 2.4.7 pypi_0 pypi
pyrsistent 0.18.0 pypi_0 pypi
pysimdjson 4.0.2 pypi_0 pypi
pysocks 1.7.1 py38h50d1736_4 conda-forge
pytest 6.2.5 pypi_0 pypi
python 3.8.12 h88f2d9e_0
python-dateutil 2.8.2 pypi_0 pypi
python-jsonschema-objects 0.4.0 pypi_0 pypi
python_abi 3.8 2_cp38 conda-forge
pytz 2021.3 pypi_0 pypi
pywavelets 1.1.1 pypi_0 pypi
readline 8.1 h9ed2024_0
reproc 14.2.3 h0d85af4_0 conda-forge
reproc-cpp 14.2.3 he49afe7_0 conda-forge
requests 2.26.0 pyhd8ed1ab_0 conda-forge
rsa 4.7.2 pypi_0 pypi
ruamel_yaml 0.15.80 py38h96a0964_1006 conda-forge
s3transfer 0.5.0 pypi_0 pypi
scikit-image 0.18.3 pypi_0 pypi
scikit-learn 1.0.1 pypi_0 pypi
scipy 1.7.1 pypi_0 pypi
setuptools 58.0.4 py38hecd8cb5_0
simplejpeg 1.6.2 pypi_0 pypi
six 1.16.0 pyh6c4a22f_0 conda-forge
sockjs-tornado 1.0.7 pypi_0 pypi
sqlite 3.36.0 hce871da_0
tenacity 8.0.1 pypi_0 pypi
threadpoolctl 3.0.0 pypi_0 pypi
tifffile 2021.11.2 pypi_0 pypi
tinybrain 1.2.1 pypi_0 pypi
tk 8.6.11 h7bc2e8c_0
toml 0.10.2 pypi_0 pypi
tornado 6.1 pypi_0 pypi
tqdm 4.62.3 pyhd8ed1ab_0 conda-forge
urllib3 1.26.7 pyhd8ed1ab_0 conda-forge
wheel 0.37.0 pyhd3eb1b0_1
xz 5.2.5 h1de35cc_0
yaml 0.2.5 haf1e3a3_0 conda-forge
z5py 2.0.9 py38h2720e7c_0 conda-forge
zlib 1.2.11 h1de35cc_3
zmesh 0.5.0 pypi_0 pypi
zope-event 4.5.0 pypi_0 pypi
zope-interface 5.4.0 pypi_0 pypi
zstandard 0.16.0 pypi_0 pypi
zstd 1.5.0 h582d3a0_0 conda-forge
It looks like this mixing packages from defaults and conda-forge in an incompatible way. That is likely leading to these symbol loading errors. Would set strict channel priority and create a fresh environment
@jakirkham I am currently trying to build using std::filesystem instead of boost::filesystem to get rid of the boost linking all together and make z5py easier to install.
However, the build is failing, I think due to mac builds with c++14, see conda-forge/z5py-feedstock#47 (comment). Could you maybe have a look in the issue? Thanks!
@jakirkham the default you mentioned is the pypi?
So I should install all the packages from conda-forge rather than mixing with pypi?
No defaults is a different channel. If you run conda config --set show_channel_urls true, then it will show up on the lines that currently don't list a channel.
I created a fresh environment but still got the same symbol error.
>>> import z5py
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/__init__.py", line 1, in <module>
from .file import File, N5File, ZarrFile, S3File
File "/Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/file.py", line 6, in <module>
from . import _z5py
ImportError: dlopen(/Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so, 2): Symbol not found: __ZN5boost10filesystem4path25m_path_iterator_incrementERNS1_8iteratorE
Referenced from: /Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so
Expected in: /Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/../../../libboost_filesystem.dylib
in /Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so
all the packages are from conda-forge
# packages in environment at /Users/jwu/opt/anaconda3/envs/z5py:
#
# Name Version Build Channel
blosc 1.21.0 he49afe7_0 conda-forge
boost-cpp 1.77.0 hf3dc895_1 conda-forge
bzip2 1.0.8 h0d85af4_4 conda-forge
ca-certificates 2021.10.8 h033912b_0 conda-forge
icu 69.1 he49afe7_0 conda-forge
libblas 3.9.0 12_osx64_openblas conda-forge
libcblas 3.9.0 12_osx64_openblas conda-forge
libcxx 12.0.1 habf9029_0 conda-forge
libffi 3.4.2 he49afe7_4 conda-forge
libgfortran 5.0.0 9_3_0_h6c81a4c_23 conda-forge
libgfortran5 9.3.0 h6c81a4c_23 conda-forge
liblapack 3.9.0 12_osx64_openblas conda-forge
libopenblas 0.3.18 openmp_h3351f45_0 conda-forge
libzlib 1.2.11 h9173be1_1013 conda-forge
llvm-openmp 12.0.1 hda6cdc1_1 conda-forge
lz4-c 1.9.3 he49afe7_1 conda-forge
ncurses 6.2 h2e338ed_4 conda-forge
numpy 1.21.3 py39h7eed0ac_1 conda-forge
openssl 3.0.0 h0d85af4_2 conda-forge
pip 21.3.1 pyhd8ed1ab_0 conda-forge
python 3.9.7 h38b4d05_3_cpython conda-forge
python_abi 3.9 2_cp39 conda-forge
readline 8.1 h05e3726_0 conda-forge
setuptools 58.5.3 py39h6e9494a_0 conda-forge
sqlite 3.36.0 h23a322b_2 conda-forge
tk 8.6.11 h5dbffcc_1 conda-forge
tzdata 2021e he74cb21_0 conda-forge
wheel 0.37.0 pyhd8ed1ab_1 conda-forge
xz 5.2.5 haf1e3a3_1 conda-forge
z5py 2.0.9 py39hdfa2d75_0 conda-forge
zlib 1.2.11 h9173be1_1013 conda-forge
zstd 1.5.0 h582d3a0_0 conda-forge
My guess is that the boost version is incompatible with the one z5py was built with. In theory conda-forge should pin the correct version, but maybe this fails here?
I am trying to update the conda-forge build to use std::filesystem, so that linking to boost is not necessary any longer in conda-forge/z5py-feedstock#47.
However, the build fails, despite working in the CI in this repository. Any help there would be highly appreciated.
@jingpengw there is a z5py version available on conda-forge now that does not rely on boost.
Could you try installing z5py>=2.0.11 and see if that fixes the issue?
can not find this version.
PackagesNotFoundError: The following packages are not available from current channels:
- z5py=2.0.11
Current channels:
- https://conda.anaconda.org/conda-forge/osx-64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/osx-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/osx-64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/osx-64
- https://repo.anaconda.com/pkgs/r/noarch
It's listed here: https://anaconda.org/conda-forge/z5py/files. But was only added ~ 30 minutes ago.
It sometimes takes a while for new packages to become available fully (not sure why, maybe some caches need to be updated). Could you please try again tomorrow or next week and report back?
conda-forge mirrors packages to a CDN (hence the delay)
I just tried, it works! thanks a lot! @constantinpape @jakirkham