_pwl_calibration_ops.so image not found
gyz0807-ai opened this issue · comments
I just installed tensorflow-lattice on a MacOS but got the following importing error. Do you know what's happening here?
Python 3.6.8 |Anaconda, Inc.| (default, Dec 29 2018, 19:04:46)
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
import tensorflow_lattice
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Traceback (most recent call last):
File "", line 1, in
File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/init.py", line 33, in
from tensorflow_lattice.python.estimators.calibrated import input_calibration_layer_from_hparams
File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/estimators/calibrated.py", line 28, in
from tensorflow_lattice.python.lib import pwl_calibration_layers
File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/lib/pwl_calibration_layers.py", line 36, in
from tensorflow_lattice.python.ops import pwl_calibration_ops
File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/ops/pwl_calibration_ops.py", line 45, in
'../../cc/ops/_pwl_calibration_ops.so'))
File "/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/anaconda3/envs/tf_lattice/lib/python3.6/site-packages/tensorflow_lattice/python/ops/../../cc/ops/_pwl_calibration_ops.so, 6): image not found
Could it be the issue here:
https://stackoverflow.com/questions/43485569/installed-a-package-with-anaconda-cant-import-in-python
If that is not the case, then can you please share a bit more detail about your python setup and which version of the pip package you installed?
I got the same issue. Not the stackoverflow issue. The _pwl_calibration_ops.so file was missing in the python3.6/site-packages/tensorflow_lattice/python/ops/../../cc/ops/ directory.
@mmilanifard Thanks for the reply. It's not the case since I was using Python installed in the virtual env. I was using Python 3.6.8 and pip 19.2.2.
@mmilanifard same issue. Running on python 3.6.0, tensorflow 1.14.0, pip 19.2.2. Additionally, I am using pyenv to manage multiple python environments on MacOS. I guess this issue is arising due to conflicting $PATH for python that is in use, but I have no clue how to resolve it. Here's the error, same as @gyz0807-ai
NotFoundError Traceback (most recent call last)
<ipython-input-1-0adc1b68d24f> in <module>
----> 1 import tensorflow_lattice
~/lattice/lib/python3.6/site-packages/tensorflow_lattice/__init__.py in <module>
31 # Import all modules here, but only import functions and classes that are
32 # more likely to be used directly by users.
---> 33 from tensorflow_lattice.python.estimators.calibrated import input_calibration_layer_from_hparams
34 from tensorflow_lattice.python.estimators.calibrated_etl import calibrated_etl_classifier
35 from tensorflow_lattice.python.estimators.calibrated_etl import calibrated_etl_regressor
~/lattice/lib/python3.6/site-packages/tensorflow_lattice/python/estimators/calibrated.py in <module>
26 from tensorflow_lattice.python.estimators import hparams as tf_lattice_hparams
27 from tensorflow_lattice.python.lib import keypoints_initialization
---> 28 from tensorflow_lattice.python.lib import pwl_calibration_layers
29 from tensorflow_lattice.python.lib import regularizers
30 from tensorflow_lattice.python.lib import tools
~/lattice/lib/python3.6/site-packages/tensorflow_lattice/python/lib/pwl_calibration_layers.py in <module>
34 from tensorflow_lattice.python.lib import regularizers
35 from tensorflow_lattice.python.lib import tools
---> 36 from tensorflow_lattice.python.ops import pwl_calibration_ops
37
38
~/lattice/lib/python3.6/site-packages/tensorflow_lattice/python/ops/pwl_calibration_ops.py in <module>
43 _pwl_calibration_ops = tf.load_op_library(
44 tf.compat.v1.resource_loader.get_path_to_datafile(
---> 45 '../../cc/ops/_pwl_calibration_ops.so'))
46
47
~/lattice/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py in load_op_library(library_filename)
59 RuntimeError: when unable to load the library or get the python wrappers.
60 """
---> 61 lib_handle = py_tf.TF_LoadLibrary(library_filename)
62
63 op_list_str = py_tf.TF_GetOpList(lib_handle)
NotFoundError: dlopen(/Users/namanshukla/lattice/lib/python3.6/site-packages/tensorflow_lattice/python/ops/../../cc/ops/_pwl_calibration_ops.so, 6): image not found
This is almost certainly a PATH/PYTHONPATH issue and I assume is related to multi python setup with virtual envs. The wheel includes the so files:
Can you check that these files are present after the pip install? You can change the PYTHONPATH if needed:
https://stackoverflow.com/questions/4757178/how-do-you-set-your-pythonpath-in-an-already-created-virtualenv
_lattice_ops.so and _pwl_calibration_ops.so files are missing for me. I tried running it after copying these files manually but still the same error.
I think the issue is with python version 3.6. tensorflow-lattice requires python3.7. I found that from https://pypi.org/project/tensorflow-lattice/#files , where the wheel was built for python3.7 (tensorflow_lattice-0.9.9-cp37-cp37m-macosx_10_9_x86_64.whl)
The following conda env set up works for me
conda create -n tensorflow-lattice -c anaconda python=3.7
source activate tensorflow-lattice
pip install tensorflow-lattice
pip install pandas #for running examples
Thanks @pbalapra for sharing your solution. Can others try to see if using py3.7 solves their problem?
@mmilanifard I just confirmed that Python 3.7 + tensorflow lattice worked. Thanks all for the help. I think we can clarify this in the documentation.
Thanks for checking. Will update the docs. Marking as closed.