Giters

thustorage / octopus

An RDMA-enabled Distributed Persistent Memory File System

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Linkage Error

agnesnatasya opened this issue · comments

commented

Environment

Ubuntu 18.04

Problem

When building Octopus, I always encounter a linking error for the NRFS file system

CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `___ibv_query_port':
/usr/include/infiniband/verbs.h:2263: undefined reference to `ibv_query_port'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::CreateResources()':
/data/octopus/src/net/RdmaSocket.cpp:77: undefined reference to `ibv_get_device_list'
/data/octopus/src/net/RdmaSocket.cpp:93: undefined reference to `ibv_get_device_name'
/data/octopus/src/net/RdmaSocket.cpp:95: undefined reference to `ibv_get_device_name'
/data/octopus/src/net/RdmaSocket.cpp:107: undefined reference to `ibv_open_device'
/data/octopus/src/net/RdmaSocket.cpp:114: undefined reference to `ibv_free_device_list'
/data/octopus/src/net/RdmaSocket.cpp:126: undefined reference to `ibv_create_cq'
/data/octopus/src/net/RdmaSocket.cpp:136: undefined reference to `ibv_alloc_pd'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `__ibv_reg_mr':
/usr/include/infiniband/verbs.h:2409: undefined reference to `ibv_reg_mr'
/usr/include/infiniband/verbs.h:2412: undefined reference to `ibv_reg_mr_iova2'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::CreateResources()':
/data/octopus/src/net/RdmaSocket.cpp:185: undefined reference to `ibv_destroy_cq'
/data/octopus/src/net/RdmaSocket.cpp:188: undefined reference to `ibv_dealloc_pd'
/data/octopus/src/net/RdmaSocket.cpp:192: undefined reference to `ibv_close_device'
/data/octopus/src/net/RdmaSocket.cpp:196: undefined reference to `ibv_free_device_list'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::CreateQueuePair(PeerSockData*, int)':
/data/octopus/src/net/RdmaSocket.cpp:248: undefined reference to `ibv_create_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ModifyQPtoInit(ibv_qp*)':
/data/octopus/src/net/RdmaSocket.cpp:274: undefined reference to `ibv_modify_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ModifyQPtoRTR(ibv_qp*, unsigned int, unsigned short, unsigned char*)':
/data/octopus/src/net/RdmaSocket.cpp:314: undefined reference to `ibv_modify_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ModifyQPtoRTS(ibv_qp*)':
/data/octopus/src/net/RdmaSocket.cpp:341: undefined reference to `ibv_modify_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ConnectQueuePair(PeerSockData*)':
/data/octopus/src/net/RdmaSocket.cpp:395: undefined reference to `ibv_query_gid'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ResourcesDestroy()':
/data/octopus/src/net/RdmaSocket.cpp:495: undefined reference to `ibv_destroy_qp'
/data/octopus/src/net/RdmaSocket.cpp:514: undefined reference to `ibv_dereg_mr'
/data/octopus/src/net/RdmaSocket.cpp:521: undefined reference to `ibv_dealloc_pd'
/data/octopus/src/net/RdmaSocket.cpp:528: undefined reference to `ibv_close_device'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::PollCompletion(unsigned short, int, ibv_wc*)':
/data/octopus/src/net/RdmaSocket.cpp:1148: undefined reference to `ibv_wc_status_str'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::PollWithCQ(int, int, ibv_wc*)':
/data/octopus/src/net/RdmaSocket.cpp:1171: undefined reference to `ibv_wc_status_str'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::PollOnce(int, int, ibv_wc*)':
/data/octopus/src/net/RdmaSocket.cpp:1189: undefined reference to `ibv_wc_status_str'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::RdmaQueryQueuePair(unsigned short)':
/data/octopus/src/net/RdmaSocket.cpp:1217: undefined reference to `ibv_query_qp'
collect2: error: ld returned 1 exit status
CMakeFiles/dmfs.dir/build.make:433: recipe for target 'dmfs' failed
make[2]: *** [dmfs] Error 1
CMakeFiles/Makefile2:215: recipe for target 'CMakeFiles/dmfs.dir/all' failed
make[1]: *** [CMakeFiles/dmfs.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 78%] Built target jninrfs
[ 80%] Linking CXX shared library libnrfsc.so
[ 80%] Built target nrfsc
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `___ibv_query_port':
/usr/include/infiniband/verbs.h:2263: undefined reference to `ibv_query_port'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::CreateResources()':
/data/octopus/src/net/RdmaSocket.cpp:77: undefined reference to `ibv_get_device_list'
/data/octopus/src/net/RdmaSocket.cpp:93: undefined reference to `ibv_get_device_name'
/data/octopus/src/net/RdmaSocket.cpp:95: undefined reference to `ibv_get_device_name'
/data/octopus/src/net/RdmaSocket.cpp:107: undefined reference to `ibv_open_device'
/data/octopus/src/net/RdmaSocket.cpp:114: undefined reference to `ibv_free_device_list'
/data/octopus/src/net/RdmaSocket.cpp:126: undefined reference to `ibv_create_cq'
/data/octopus/src/net/RdmaSocket.cpp:136: undefined reference to `ibv_alloc_pd'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `__ibv_reg_mr':
/usr/include/infiniband/verbs.h:2409: undefined reference to `ibv_reg_mr'
/usr/include/infiniband/verbs.h:2412: undefined reference to `ibv_reg_mr_iova2'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::CreateResources()':
/data/octopus/src/net/RdmaSocket.cpp:185: undefined reference to `ibv_destroy_cq'
/data/octopus/src/net/RdmaSocket.cpp:188: undefined reference to `ibv_dealloc_pd'
/data/octopus/src/net/RdmaSocket.cpp:192: undefined reference to `ibv_close_device'
/data/octopus/src/net/RdmaSocket.cpp:196: undefined reference to `ibv_free_device_list'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::CreateQueuePair(PeerSockData*, int)':
/data/octopus/src/net/RdmaSocket.cpp:248: undefined reference to `ibv_create_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ModifyQPtoInit(ibv_qp*)':
/data/octopus/src/net/RdmaSocket.cpp:274: undefined reference to `ibv_modify_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ModifyQPtoRTR(ibv_qp*, unsigned int, unsigned short, unsigned char*)':
/data/octopus/src/net/RdmaSocket.cpp:314: undefined reference to `ibv_modify_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ModifyQPtoRTS(ibv_qp*)':
/data/octopus/src/net/RdmaSocket.cpp:341: undefined reference to `ibv_modify_qp'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ConnectQueuePair(PeerSockData*)':
/data/octopus/src/net/RdmaSocket.cpp:395: undefined reference to `ibv_query_gid'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::ResourcesDestroy()':
/data/octopus/src/net/RdmaSocket.cpp:495: undefined reference to `ibv_destroy_qp'
/data/octopus/src/net/RdmaSocket.cpp:514: undefined reference to `ibv_dereg_mr'
/data/octopus/src/net/RdmaSocket.cpp:521: undefined reference to `ibv_dealloc_pd'
/data/octopus/src/net/RdmaSocket.cpp:528: undefined reference to `ibv_close_device'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::PollCompletion(unsigned short, int, ibv_wc*)':
/data/octopus/src/net/RdmaSocket.cpp:1148: undefined reference to `ibv_wc_status_str'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::PollWithCQ(int, int, ibv_wc*)':
/data/octopus/src/net/RdmaSocket.cpp:1171: undefined reference to `ibv_wc_status_str'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::PollOnce(int, int, ibv_wc*)':
/data/octopus/src/net/RdmaSocket.cpp:1189: undefined reference to `ibv_wc_status_str'
CMakeFiles/dmfs.dir/src/net/RdmaSocket.cpp.o: In function `RdmaSocket::RdmaQueryQueuePair(unsigned short)':
/data/octopus/src/net/RdmaSocket.cpp:1217: undefined reference to `ibv_query_qp'
collect2: error: ld returned 1 exit status
CMakeFiles/dmfs.dir/build.make:433: recipe for target 'dmfs' failed
make[2]: *** [dmfs] Error 1
CMakeFiles/Makefile2:215: recipe for target 'CMakeFiles/dmfs.dir/all' failed
make[1]: *** [CMakeFiles/dmfs.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 78%] Built target jninrfs
[ 80%] Linking CXX shared library libnrfsc.so
[ 80%] Built target nrfsc
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Checking ldconfig gives me this result

$ ldconfig -p | grep libibverbs
libibverbs.so.1 (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libibverbs.so.1
libibverbs.so (libc6,x86-64) => /usr/lib/x86_64-linux-gnu/libibverbs.so

Have this error been encountered before? Would appreciate your help for the problem above! Thank you!

commented

I managed to solve it by manually linking the libraries and adding these lines in the CMakeLists.txt!

link_libraries("ibverbs")
link_libraries("fuse")
commented

I managed to solve it by manually linking the libraries and adding these lines in the CMakeLists.txt!

link_libraries("ibverbs")
link_libraries("fuse")

thanks！