Error:sequential_batch_fft.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
13052 opened this issue · comments
Hi, when I run sh train_dist_faster.sh on Ubuntu 16.04&Python 3.6&tensorflow1.8, I got this:
+ set -e
+ export PYTHONUNBUFFERED=True
+ PYTHONUNBUFFERED=True
+ GPU_ID=0
+ DATASET=coco
+ NET=res101
+ array='0 coco res101 EXP_DIR coco_flip_0001_bilinear_new'
+ len=1
+ EXTRA_ARGS=
+ EXTRA_ARGS_SLUG=
+ case ${DATASET} in
+ TRAIN_IMDB=train_filter_single.txt
+ TEST_IMDB=test_filter_single.txt
+ STEPSIZE=40000
+ ITERS=110000
+ ANCHORS='[8,16,32,64]'
+ RATIOS='[0.5,1,2]'
++ date +%Y-%m-%d_%H-%M-%S
+ LOG=./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
+ exec
++ tee -a ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
+ echo Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
+ set +x
+ '[' '!' -f output/res101/train_filter_single.txt/default/res101_faster_rcnn_iter_110000.ckpt.index ']'
+ [[ ! -z '' ]]
+ python ./tools/trainval_net.py --weight /home/yihang/detect_ps/RGB-N/models/res101.ckpt --imdb /home/yihang/detect_ps/RGB-N/coco_synthetic/train_filter_single.txt --imdbval /home/yihang/detect_ps/RGB-N/coco_synthetic/test_filter_single.txt --iters 110000 --cfg cfgs/res101.yml --net res101 --set ANCHOR_SCALES '[8,16,32,64]' ANCHOR_RATIOS '[0.5,1,2]' TRAIN.STEPSIZE 40000
Traceback (most recent call last):
File "./tools/trainval_net.py", line 25, in <module>
from nets.resnet_v1_noise import resnet_noise
File "/home/yihang/detect_ps/RGB-N/tools/../lib/nets/resnet_v1_noise.py", line 25, in <module>
from compact_bilinear_pooling.compact_bilinear_pooling import compact_bilinear_pooling_layer
File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/compact_bilinear_pooling.py", line 6, in <module>
from .sequential_fft import sequential_batch_fft, sequential_batch_ifft
File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/__init__.py", line 1, in <module>
from .sequential_batch_fft_ops import sequential_batch_fft, sequential_batch_ifft
File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/sequential_batch_fft_ops.py", line 10, in <module>
'build/sequential_batch_fft.so'))
File "/home/yihang/anaconda3/envs/py36tf13/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/build/sequential_batch_fft.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
Anyone can help?
Hi, when I run
sh train_dist_faster.shon Ubuntu 16.04&Python 3.6&tensorflow1.8, I got this:+ set -e + export PYTHONUNBUFFERED=True + PYTHONUNBUFFERED=True + GPU_ID=0 + DATASET=coco + NET=res101 + array='0 coco res101 EXP_DIR coco_flip_0001_bilinear_new' + len=1 + EXTRA_ARGS= + EXTRA_ARGS_SLUG= + case ${DATASET} in + TRAIN_IMDB=train_filter_single.txt + TEST_IMDB=test_filter_single.txt + STEPSIZE=40000 + ITERS=110000 + ANCHORS='[8,16,32,64]' + RATIOS='[0.5,1,2]' ++ date +%Y-%m-%d_%H-%M-%S + LOG=./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + exec ++ tee -a ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + echo Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + set +x + '[' '!' -f output/res101/train_filter_single.txt/default/res101_faster_rcnn_iter_110000.ckpt.index ']' + [[ ! -z '' ]] + python ./tools/trainval_net.py --weight /home/yihang/detect_ps/RGB-N/models/res101.ckpt --imdb /home/yihang/detect_ps/RGB-N/coco_synthetic/train_filter_single.txt --imdbval /home/yihang/detect_ps/RGB-N/coco_synthetic/test_filter_single.txt --iters 110000 --cfg cfgs/res101.yml --net res101 --set ANCHOR_SCALES '[8,16,32,64]' ANCHOR_RATIOS '[0.5,1,2]' TRAIN.STEPSIZE 40000 Traceback (most recent call last): File "./tools/trainval_net.py", line 25, in <module> from nets.resnet_v1_noise import resnet_noise File "/home/yihang/detect_ps/RGB-N/tools/../lib/nets/resnet_v1_noise.py", line 25, in <module> from compact_bilinear_pooling.compact_bilinear_pooling import compact_bilinear_pooling_layer File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/compact_bilinear_pooling.py", line 6, in <module> from .sequential_fft import sequential_batch_fft, sequential_batch_ifft File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/__init__.py", line 1, in <module> from .sequential_batch_fft_ops import sequential_batch_fft, sequential_batch_ifft File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/sequential_batch_fft_ops.py", line 10, in <module> 'build/sequential_batch_fft.so')) File "/home/yihang/anaconda3/envs/py36tf13/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/build/sequential_batch_fft.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEvAnyone can help?
Hi,
Did you compile the bilinear pooling before run the script? Also, the original code is run on tensorflow 0.12 and 1.8 will have some issues due to the compatibility.
Hi, when I run
sh train_dist_faster.shon Ubuntu 16.04&Python 3.6&tensorflow1.8, I got this:+ set -e + export PYTHONUNBUFFERED=True + PYTHONUNBUFFERED=True + GPU_ID=0 + DATASET=coco + NET=res101 + array='0 coco res101 EXP_DIR coco_flip_0001_bilinear_new' + len=1 + EXTRA_ARGS= + EXTRA_ARGS_SLUG= + case ${DATASET} in + TRAIN_IMDB=train_filter_single.txt + TEST_IMDB=test_filter_single.txt + STEPSIZE=40000 + ITERS=110000 + ANCHORS='[8,16,32,64]' + RATIOS='[0.5,1,2]' ++ date +%Y-%m-%d_%H-%M-%S + LOG=./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + exec ++ tee -a ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + echo Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + set +x + '[' '!' -f output/res101/train_filter_single.txt/default/res101_faster_rcnn_iter_110000.ckpt.index ']' + [[ ! -z '' ]] + python ./tools/trainval_net.py --weight /home/yihang/detect_ps/RGB-N/models/res101.ckpt --imdb /home/yihang/detect_ps/RGB-N/coco_synthetic/train_filter_single.txt --imdbval /home/yihang/detect_ps/RGB-N/coco_synthetic/test_filter_single.txt --iters 110000 --cfg cfgs/res101.yml --net res101 --set ANCHOR_SCALES '[8,16,32,64]' ANCHOR_RATIOS '[0.5,1,2]' TRAIN.STEPSIZE 40000 Traceback (most recent call last): File "./tools/trainval_net.py", line 25, in <module> from nets.resnet_v1_noise import resnet_noise File "/home/yihang/detect_ps/RGB-N/tools/../lib/nets/resnet_v1_noise.py", line 25, in <module> from compact_bilinear_pooling.compact_bilinear_pooling import compact_bilinear_pooling_layer File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/compact_bilinear_pooling.py", line 6, in <module> from .sequential_fft import sequential_batch_fft, sequential_batch_ifft File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/__init__.py", line 1, in <module> from .sequential_batch_fft_ops import sequential_batch_fft, sequential_batch_ifft File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/sequential_batch_fft_ops.py", line 10, in <module> 'build/sequential_batch_fft.so')) File "/home/yihang/anaconda3/envs/py36tf13/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/build/sequential_batch_fft.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEvAnyone can help?
Hi,
Did you compile the bilinear pooling before run the script? Also, the original code is run on tensorflow 0.12 and 1.8 will have some issues due to the compatibility.
I compiled the bilinear pooling several times, and with some tricks of searching results to avoid this error but never worked, that's why I came here. Anyway, now I think the best way to address this issue is creating an environment just like your original repo. Thanks!
Hi, when I run
sh train_dist_faster.shon Ubuntu 16.04&Python 3.6&tensorflow1.8, I got this:+ set -e + export PYTHONUNBUFFERED=True + PYTHONUNBUFFERED=True + GPU_ID=0 + DATASET=coco + NET=res101 + array='0 coco res101 EXP_DIR coco_flip_0001_bilinear_new' + len=1 + EXTRA_ARGS= + EXTRA_ARGS_SLUG= + case ${DATASET} in + TRAIN_IMDB=train_filter_single.txt + TEST_IMDB=test_filter_single.txt + STEPSIZE=40000 + ITERS=110000 + ANCHORS='[8,16,32,64]' + RATIOS='[0.5,1,2]' ++ date +%Y-%m-%d_%H-%M-%S + LOG=./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + exec ++ tee -a ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + echo Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26 + set +x + '[' '!' -f output/res101/train_filter_single.txt/default/res101_faster_rcnn_iter_110000.ckpt.index ']' + [[ ! -z '' ]] + python ./tools/trainval_net.py --weight /home/yihang/detect_ps/RGB-N/models/res101.ckpt --imdb /home/yihang/detect_ps/RGB-N/coco_synthetic/train_filter_single.txt --imdbval /home/yihang/detect_ps/RGB-N/coco_synthetic/test_filter_single.txt --iters 110000 --cfg cfgs/res101.yml --net res101 --set ANCHOR_SCALES '[8,16,32,64]' ANCHOR_RATIOS '[0.5,1,2]' TRAIN.STEPSIZE 40000 Traceback (most recent call last): File "./tools/trainval_net.py", line 25, in <module> from nets.resnet_v1_noise import resnet_noise File "/home/yihang/detect_ps/RGB-N/tools/../lib/nets/resnet_v1_noise.py", line 25, in <module> from compact_bilinear_pooling.compact_bilinear_pooling import compact_bilinear_pooling_layer File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/compact_bilinear_pooling.py", line 6, in <module> from .sequential_fft import sequential_batch_fft, sequential_batch_ifft File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/__init__.py", line 1, in <module> from .sequential_batch_fft_ops import sequential_batch_fft, sequential_batch_ifft File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/sequential_batch_fft_ops.py", line 10, in <module> 'build/sequential_batch_fft.so')) File "/home/yihang/anaconda3/envs/py36tf13/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library lib_handle = py_tf.TF_LoadLibrary(library_filename) tensorflow.python.framework.errors_impl.NotFoundError: /home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/build/sequential_batch_fft.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEvAnyone can help?
Have you solved the problem?
Maybe you should include Tensorflow lib path to the g++ command in compile.sh file. For example: -L{Path to tensorflow.so} -ltensorflow_framework