Build in Docker Dynamic Library librpma.so.0 Not Found
DanielLee343 opened this issue · comments
Hi, I was using the Dockerfile.debian-stable to build librpma inside docker. I start the container with the command:
docker run --network=bridge --shm-size=4G -v /root/yuze/rpma:/opt/workspace:z -v /mnt/pmemfs1:/mnt/pmemfs1:z -w /opt/workspace/ -p 1234:1234 -e PKG_CONFIG_PATH=/opt/pmdk/lib/pkgconfig -it rpma:debian-stable /bin/bash
As you can see I mounted the rpma repo to /opt/workspace within docker. Then inside docker I performed:
mkdir build && cd build
cmake ..
make -j
sudo make install
And it worked fine
But when I tried to run the example
root@ced9f8a699ba:/opt/workspace/examples/03-read-to-persistent# ./server 192.168.0.12 1234 /mnt/pmemfs1/testio
./server: error while loading shared libraries: librpma.so.0: cannot open shared object file: No such file or directory
root@ced9f8a699ba:/opt/workspace/examples/03-read-to-persistent# ldd ./server
linux-vdso.so.1 (0x00007ffce3dfb000)
librpma.so.0 => not found
libibverbs.so.1 => /usr/lib/x86_64-linux-gnu/libibverbs.so.1 (0x00007f88b7f79000)
libpmem.so.1 => /usr/lib/x86_64-linux-gnu/libpmem.so.1 (0x00007f88b7f24000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f88b7d5f000)
libnl-route-3.so.200 => /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200 (0x00007f88b7ce4000)
libnl-3.so.200 => /lib/x86_64-linux-gnu/libnl-3.so.200 (0x00007f88b7cc1000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f88b7c9d000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f88b7c97000)
libndctl.so.6 => /usr/lib/x86_64-linux-gnu/libndctl.so.6 (0x00007f88b7c6f000)
libdaxctl.so.1 => /usr/lib/x86_64-linux-gnu/libdaxctl.so.1 (0x00007f88b7c62000)
/lib64/ld-linux-x86-64.so.2 (0x00007f88b7fa7000)
libudev.so.1 => /usr/lib/x86_64-linux-gnu/libudev.so.1 (0x00007f88b7c3a000)
libuuid.so.1 => /usr/lib/x86_64-linux-gnu/libuuid.so.1 (0x00007f88b7c2f000)
libkmod.so.2 => /usr/lib/x86_64-linux-gnu/libkmod.so.2 (0x00007f88b7c13000)
liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007f88b7beb000)
libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f88b78f7000)
Seems the dynamic linking librpma.so.0 is not found. Perhaps it's not the way to build rpma inside docker? Thanks!
Hi, it works on GitHub Actions on Debian Stable in our CI.
See:
- branch: https://github.com/ldorau/rpma/tree/test-run_ldd_on_examples-on-Debian-Stable
- commit: ldorau@2331c2f
- build: https://github.com/ldorau/rpma/runs/7088805264
Read the log of the last build (no. 6) at https://github.com/ldorau/rpma/runs/7088805264.
ldd is run every time on all server and client binaries and they are always correctly linked:
+ grep -e librpma.so
+ ldd ./examples/03-read-to-persistent/server
librpma.so.0 => /rpma/build/src/librpma.so.0 (0x00007f0cbc29a000)
There is also the full docker run command printed out in this log.
Please check it and try out yourself.
You can also list all commands you have run, so we can try to reproduce your issue.
Hi @ldorau, thanks for your branch. I forked from your branch to mine and followed the command. Then I had this permission denied error inside container. See all commands I ran below:
Clone the git repo:
[lyuze@storage-2 rpma_docker]$ cat /etc/os-release
NAME="CentOS Stream"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Stream 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8"
REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream"
[lyuze@storage-2 rpma_docker]$ git version
git version 2.31.1
[lyuze@storage-2 rpma_docker]$ git lfs version
git-lfs/3.2.0 (GitHub; linux amd64; go 1.18.2)
[lyuze@storage-2 rpma_docker]$ git init "/root/yuze/rpma_docker/rpma"
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /root/yuze/rpma_docker/rpma/.git/
[lyuze@storage-2 rpma_docker]$ cd rpma
[lyuze@storage-2 rpma]$ git remote add origin https://github.com/DanielLee343/rpma
[lyuze@storage-2 rpma]$ git config gc.auto 0
[lyuze@storage-2 rpma]$ git config --get-all http.https://github.com/DanielLee343/rpma.extraheader
[lyuze@storage-2 rpma]$ git -c http.extraheader="AUTHORIZATION: basic ***" fetch --tags --prune --progress --no-recurse-submodules origin +refs/heads/*:refs/remotes/origin/*
Warning: Permanently added the ECDSA host key for IP address '140.82.112.3' to the list of known hosts.
remote: Enumerating objects: 19928, done.
remote: Total 19928 (delta 0), reused 0 (delta 0), pack-reused 19928
Receiving objects: 100% (19928/19928), 4.10 MiB | 28.75 MiB/s, done.
Resolving deltas: 100% (14483/14483), done.
From github.com:DanielLee343/rpma
* [new branch] CI-base -> origin/CI-base
* [new branch] OLD-test-minUSSR-devel -> origin/OLD-test-minUSSR-devel
* [new branch] RUN-TRAVIS -> origin/RUN-TRAVIS
* [new branch] Revert-tools-fix-bad-whitespace-errors -> origin/Revert-tools-fix-bad-whitespace-errors
* [new branch] bench -> origin/bench
* [new branch] circleci-update-resource-class -> origin/circleci-update-resource-class
* [new branch] common-LDORAU-force-rebuild-and-push-new-images -> origin/common-LDORAU-force-rebuild-and-push-new-images
* ...
[lyuze@storage-2 rpma]$ git checkout --progress --force 2331c2f0c89d9e90f052e2d05951605fd0f77a5e
Note: switching to '2331c2f0c89d9e90f052e2d05951605fd0f77a5e'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 2331c2f0 test: run_ldd_on_examples on Debian Stable
Pull or rebuild the image:
[lyuze@storage-2 rpma]$ echo $GITHUB_REPO
pmem/rpma
[lyuze@storage-2 rpma]$ echo $GH_CR_ADDR
ghcr.io
[lyuze@storage-2 rpma]$ echo $DOCKER_REPO
ghcr.io/pmem/rpma
[lyuze@storage-2 rpma]$ export GH_CR_PAT=<myToken>
[lyuze@storage-2 rpma]$ echo $GH_CR_PAT
<myToken>
[lyuze@storage-2 rpma]$ export CR_PAT=<myToken>
[lyuze@storage-2 rpma]$ echo $CR_PAT | docker login ghcr.io -u DanielLee343 --password-stdin
Login Succeeded
[lyuze@storage-2 rpma]$ export GH_CR_USER=DanielLee343
[lyuze@storage-2 rpma]$ echo $HOST_WORKDIR
/root/yuze/rpma_docker/rpma
[lyuze@storage-2 rpma]$ echo $WORKDIR
utils/docker
[lyuze@storage-2 rpma]$ cd $WORKDIR && N=6 OS=debian OS_VER=stable TYPE=normal CC=gcc PUSH_IMAGE=1 ./pull-or-rebuild-image.sh
CI_COMMIT=2331c2f0c89d9e90f052e2d05951605fd0f77a5e
CI_COMMIT_RANGE=309af437fe30a14e158d83a404db9d7e091bee04..HEAD
CI_BRANCH=
CI_EVENT_TYPE=
CI_REPO_SLUG=
CI_CPU_ARCH=x86_64
CI_SANITS=
IMG_VER=master
Commits in the commit range:
2331c2f0c89d9e90f052e2d05951605fd0f77a5e
Files modified within the commit range:
.github/workflows/on_pull_request.yml
utils/docker/build.sh
utils/docker/run-build.sh
master-debian-stable: Pulling from pmem/rpma
d257fca7b66e: Pull complete
7ef072100652: Pull complete
fcdbedd899df: Pull complete
3a888e01ca1d: Pull complete
fe0cde4b856f: Pull complete
80c3e840869b: Pull complete
ac798a9d44cc: Pull complete
dde082c3443c: Pull complete
b1d078d49713: Pull complete
0f5dee7a3169: Pull complete
Digest: sha256:7577b041f7f75d89361918c6dedcdd5767ad6782d7555268ef6faa64323857f3
Status: Downloaded newer image for ghcr.io/pmem/rpma:master-debian-stable
ghcr.io/pmem/rpma:master-debian-stable
Run the build:
[lyuze@storage-2 rpma]$ cd $WORKDIR && N=6 OS=debian OS_VER=stable TYPE=normal CC=gcc PUSH_IMAGE=1 ./build.sh
CI_COMMIT=2331c2f0c89d9e90f052e2d05951605fd0f77a5e
CI_COMMIT_RANGE=309af437fe30a14e158d83a404db9d7e091bee04..HEAD
CI_BRANCH=
CI_EVENT_TYPE=
CI_REPO_SLUG=
CI_CPU_ARCH=x86_64
CI_SANITS=
IMG_VER=master
Building master-debian-stable
*** Running the following command: ***
$ docker run --privileged=true --name=rpma-debian-stable -i -t --env http_proxy= --env https_proxy= --env AUTO_DOC_UPDATE=0 --env GITHUB_ACTIONS= --env GITHUB_HEAD_REF= --env GITHUB_REPO=pmem/rpma --env GITHUB_REPOSITORY= --env GITHUB_REF= --env GITHUB_RUN_ID= --env GITHUB_SHA= --env WORKDIR=/rpma --env SCRIPTSDIR=/rpma/utils/docker --env TESTS_COVERAGE= --env CI_COMMIT=2331c2f0c89d9e90f052e2d05951605fd0f77a5e --env CI_COMMIT_RANGE=309af437fe30a14e158d83a404db9d7e091bee04..HEAD --env CI_REPO_SLUG= --env CI_BRANCH= --env CI_EVENT_TYPE= --env CI_RUN=NO --env CI_SANITS= --env TRAVIS= --env COVERITY_SCAN_TOKEN= --env COVERITY_SCAN_NOTIFICATION_EMAIL= --env DOC_UPDATE_GITHUB_TOKEN= --env TEST_BUILD= --env DEFAULT_TEST_DIR=/dev/shm --env TEST_PACKAGES=ON --env TESTS_PERF_TOOLS=ON --env FAULT_INJECTION= --env CC=gcc --shm-size=4G -v /root/yuze/rpma_docker/rpma:/rpma -v /etc/localtime:/etc/localtime -w /rpma/utils/docker ghcr.io/pmem/rpma:master-debian-stable ./run-build.sh
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
[sudo] password for user:
##################################################################
### Verify build with ASAN and UBSAN (gcc, DEBUG)
##################################################################
mkdir: cannot create directory '/rpma/build': Permission denied
I observed that the mkdir -p $WORKDIR/build is executed inside container. And before this, the prepare-for-build.sh is run for granting user permission under /rpma, but that's only for CIs, which I'm not. I'm I understanding correctly and is there a way to bypass? Thanks!
Hi, chown should be run always to make sure WORKDIR has correct permissions: #2007