2.37 pthread thread_pool error

Question

2.37 pthread thread_pool error

EpochalEngineer opened this issue a year ago · comments

Describe the bug

On 2.37 (just tagged, no release notes yet), the docker static build does not work on kernel 5.19.17 with Unraid 6.11.5

In file included from lib/fuse_loop.cpp:10: lib/thread_pool.hpp: In static member function 'static std::__cxx11::string ThreadPool::get_thread_name(std::__cxx11::string)': lib/thread_pool.hpp:102:5: error: 'pthread_getname_np' was not declared in this scope pthread_getname_np(pthread_self(),name,sizeof(name)); ^~~~~~~~~~~~~~~~~~ lib/thread_pool.hpp:102:5: note: suggested alternative: 'pthread_setname_np' pthread_getname_np(pthread_self(),name,sizeof(name)); ^~~~~~~~~~~~~~~~~~ pthread_setname_np lib/thread_pool.hpp: In lambda function: lib/thread_pool.hpp:197:7: error: 'pthread_getname_np' was not declared in this scope pthread_getname_np(t,name,sizeof(name)); ^~~~~~~~~~~~~~~~~~ lib/thread_pool.hpp:197:7: note: suggested alternative: 'pthread_setname_np' pthread_getname_np(t,name,sizeof(name)); ^~~~~~~~~~~~~~~~~~ pthread_setname_np

To Reproduce

Broken:
docker run -v /tmp:/build --rm -e TAG=2.37.0 trapexit/mergerfs-static-build

Works:
docker run -v /tmp:/build --rm -e TAG=2.36.0 trapexit/mergerfs-static-build

Expected behavior

Build to succeed

System information:

OS, kernel version: Linux t1001 5.19.17-Unraid # 2 SMP PREEMPT_DYNAMIC Wed Nov 2 11:54:15 PDT 2022 x86_64 AMD Ryzen 5 1600X Six-Core Processor AuthenticAMD GNU/Linux
mergerfs version: 2.37

Additional context

Add any other context about the problem here.

trapexit · Answer 1 · Wed Sep 06 2023 22:04:56 GMT+0800 (China Standard Time)

On 2.37 (just tagged, no release notes yet),

Because it hasn't been released yet. I'm still working on it but need to tag for final testing.

Can you share exactly what you're doing to build it? My static builds work fine.

edit: Sorry. skimmed too quickly. Probably old versions of musl or something. I can look at updating the builder but couldn't you just use my static bins? What's the reason for not doing so? That container isn't maintained. Just threw it together for people on a whim.

EpochalEngineer · Answer 2 · Wed Sep 06 2023 22:24:52 GMT+0800 (China Standard Time)

Ah yes, I can definitely do that. I've been using some old scripts that used that docker build method, will switch to use your static bins. Perhaps I should've read the latest readme!

With that I don't see a need to update the builder, please feel free to close this.

trapexit · Answer 3 · Wed Sep 06 2023 22:26:27 GMT+0800 (China Standard Time)

I was just looking around and found the Dockerfile for this. I can update it quick and push it so it doesn't break for others.

trapexit · Answer 4 · Wed Sep 06 2023 23:49:37 GMT+0800 (China Standard Time)

Updated. Should work now.

CurlyFlow · Answer 5 · Thu Sep 07 2023 23:36:40 GMT+0800 (China Standard Time)

Updated. Should work now.

Sadly, doenst work.

`docker run -v /mnt/user/appdata/other/mergerfs:/build --rm trapexit/mergerfs-static-build
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
v3.10.9-43-g3feb769ea3 [http://dl-cdn.alpinelinux.org/alpine/v3.10/main]
v3.10.6-10-ged79a86de3 [http://dl-cdn.alpinelinux.org/alpine/v3.10/community]
OK: 10351 distinct packages available
(1/6) Installing ca-certificates (20191127-r2)
(2/6) Installing nghttp2-libs (1.39.2-r1)
(3/6) Installing libcurl (7.66.0-r4)
(4/6) Installing expat (2.2.8-r0)
(5/6) Installing pcre2 (10.33-r0)
(6/6) Installing git (2.22.5-r0)
Executing busybox-1.30.1-r2.trigger
Executing ca-certificates-20191127-r2.trigger
OK: 21 MiB in 20 packages
Cloning into 'mergerfs'...
2.37.0
Note: checking out '2.37.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD is now at dd7e9e2 Merge pull request #1244 from trapexit/docs Installing make (4.2.1-r2)
Executing busybox-1.30.1-r2.trigger
OK: 238 MiB in 69 packages
make DEBUG= -C libfuse
make[1]: Entering directory '/tmp/mergerfs/libfuse'
mkdir -p build
touch build/stamp
ecfd/build | tee build/config.h
#ifndef CONFIG_H_INCLUDED
#define CONFIG_H_INCLUDED

#define HAVE_FORK
/tmp/mergerfs/libfuse/ecfd/tests/HAVE_MALLOC_TRIM.c: In function 'main':
/tmp/mergerfs/libfuse/ecfd/tests/HAVE_MALLOC_TRIM.c:6:3: warning: implicit declaration of function 'malloc_trim'; did you mean 'malloc'? [-Wimplicit-function-declaration]
malloc_trim(0);
^~~~~~~~~~~
malloc
/usr/lib/gcc/x86_64-alpine-linux-musl/8.3.0/../../../../x86_64-alpine-linux-musl/bin/ld: /tmp/ccBnODge.o: in function main': HAVE_MALLOC_TRIM.c:(.text+0xf): undefined reference to malloc_trim'
collect2: error: ld returned 1 exit status
#define HAVE_STRUCT_STAT_ST_ATIM
#define HAVE_UTIMENSAT

#endif
make build/buffer.o build/crc32b.o build/debug.o build/fuse.o build/fuse_dirents.o build/fuse_lowlevel.o build/node.o build/fuse_node.o build/fuse_opt.o build/fuse_session.o build/fuse_signals.o build/helper.o build/mount.o build/syslog.o build/format.o build/os.o build/cpu.o build/fuse_config.o build/fuse_loop.o build/fuse_msgbuf.o
make[2]: Entering directory '/tmp/mergerfs/libfuse'
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/buffer.c -o build/buffer.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/crc32b.c -o build/crc32b.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/debug.c -o build/debug.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse.c -o build/fuse.o
lib/fuse.c: In function 'fuse_invalidate_all_nodes':
lib/fuse.c:3902:3: warning: implicit declaration of function 'syslog_info'; did you mean 'psiginfo'? [-Wimplicit-function-declaration]
syslog_info("invalidating file entries");
^~~~~~~~~~~
psiginfo
lib/fuse.c: In function 'add_name':
lib/fuse.c:807:3: warning: 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
strncpy(s,name,len);
^~~~~~~~~~~~~~~~~~~
lib/fuse.c:781:16: note: length computed here
size_t len = strlen(name);
^~~~~~~~~~~~
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_dirents.c -o build/fuse_dirents.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_lowlevel.c -o build/fuse_lowlevel.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/node.c -o build/node.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_node.c -o build/fuse_node.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_opt.c -o build/fuse_opt.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_session.c -o build/fuse_session.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_signals.c -o build/fuse_signals.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/helper.c -o build/helper.o
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/mount.c -o build/mount.o
In file included from lib/mount_generic.c:22,
from lib/mount.c:22:
/usr/include/sys/poll.h:1:2: warning: #warning redirecting incorrect #include <sys/poll.h> to <poll.h> [-Wcpp]
#warning redirecting incorrect #include <sys/poll.h> to <poll.h>
^~~~~~~
cc -O2 -std=gnu99 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/syslog.c -o build/syslog.o
g++ -std=c++11 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/format.cpp -o build/format.o
g++ -std=c++11 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/os.cpp -o build/os.o
g++ -std=c++11 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/cpu.cpp -o build/cpu.o
g++ -std=c++11 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_config.cpp -o build/fuse_config.o
g++ -std=c++11 -Wall -pipe -MMD -Iinclude -Ibuild -D_REENTRANT -D_FILE_OFFSET_BITS=64 -DPACKAGE_VERSION="2.9.7-mergerfs_2.30.0" -DFUSERMOUNT_DIR="/usr/local/bin" -c lib/fuse_loop.cpp -o build/fuse_loop.o
In file included from lib/fuse_loop.cpp:10:
lib/thread_pool.hpp: In static member function 'static std::__cxx11::string ThreadPool::get_thread_name(std::__cxx11::string)':
lib/thread_pool.hpp:102:5: error: 'pthread_getname_np' was not declared in this scope
pthread_getname_np(pthread_self(),name,sizeof(name));
^~~~~~~~~~~~~~~~~~
lib/thread_pool.hpp:102:5: note: suggested alternative: 'pthread_setname_np'
pthread_getname_np(pthread_self(),name,sizeof(name));
^~~~~~~~~~~~~~~~~~
pthread_setname_np
lib/thread_pool.hpp: In lambda function:
lib/thread_pool.hpp:197:7: error: 'pthread_getname_np' was not declared in this scope
pthread_getname_np(t,name,sizeof(name));
^~~~~~~~~~~~~~~~~~
lib/thread_pool.hpp:197:7: note: suggested alternative: 'pthread_setname_np'
pthread_getname_np(t,name,sizeof(name));
^~~~~~~~~~~~~~~~~~
pthread_setname_np
make[2]: Leaving directory '/tmp/mergerfs/libfuse'
make[2]: *** [Makefile:126: build/fuse_loop.o] Error 1
make[1]: Leaving directory '/tmp/mergerfs/libfuse'
make[1]: *** [Makefile:105: objects] Error 2
make: *** [Makefile:257: libfuse] Error 2
strip: 'build/mergerfs': No such file
/tmp/build-mergerfs: line 18: build/mergerfs: not found
cp: can't stat 'build/mergerfs': No such file or directory`

Ah yes, I can definitely do that. I've been using some old scripts that used that docker build method, will switch to use your static bins. Perhaps I should've read the latest readme!

With that I don't see a need to update the builder, please feel free to close this.

Could you share your script?

trapexit · Answer 6 · Thu Sep 07 2023 23:43:02 GMT+0800 (China Standard Time)

Did you pull the latest version? Docker won't do that by default.

CurlyFlow · Answer 7 · Thu Sep 07 2023 23:43:48 GMT+0800 (China Standard Time)

Did you pull the latest version? Docker won't do that by default.

I just restartet my server and run the commands shown above, i though tit would be the latest then. As far as i know it downloads the latest version and removed it then. (after moving it to bin)

trapexit · Answer 8 · Thu Sep 07 2023 23:45:06 GMT+0800 (China Standard Time)

No. Reboots do not change the cached version of the container image. You must expressly pull the image. docker pull trapexit/mergerfs-static-build

salty · Answer 9 · Thu Sep 07 2023 23:45:26 GMT+0800 (China Standard Time)

Or add --pull=always to the docker run per https://docs.docker.com/engine/reference/commandline/run/

CurlyFlow · Answer 10 · Thu Sep 07 2023 23:46:53 GMT+0800 (China Standard Time)

The --rm flag is there to tell the Docker Daemon to clean up the container and remove the file system after the container exits
?

salty · Answer 11 · Thu Sep 07 2023 23:47:24 GMT+0800 (China Standard Time)

That just cleans up the container on exit, nothing to do with the image itself.

CurlyFlow · Answer 12 · Thu Sep 07 2023 23:48:02 GMT+0800 (China Standard Time)

Ah, okay, didnt know that. Cant really test it now because im happy its working again ... xD

Maybe someone know how to use that "static method" all are talking about? :D

trapexit · Answer 13 · Thu Sep 07 2023 23:51:45 GMT+0800 (China Standard Time)

Maybe someone know how to use that "static method" all are talking about? :D

I don't know what you're referring to.

CurlyFlow · Answer 14 · Thu Sep 07 2023 23:52:49 GMT+0800 (China Standard Time)

can i use

cd /tmp
wget https://github.com/trapexit/mergerfs/releases/download/2.36.0/mergerfs-static-linux_amd64.tar.gz
tar -xzvf mergerfs-static-linux_amd64.tar.gz
mv /tmp/usr/local/bin/mergerfs /bin
rm -r /tmp/usr/

thanks chatgpd? :D I just dont know if that is all i need for it to work? Just that one mergerfs file?

trapexit · Answer 15 · Fri Sep 08 2023 00:04:34 GMT+0800 (China Standard Time)

VER=2.36.0
wget -P /tmp/ https://github.com/trapexit/mergerfs/releases/download/${VER}/mergerfs-static-linux_amd64.tar.gz
tar xvf /tmp/mergerfs-static-linux_amd64.tar.gz -C /
rm /tmp/mergerfs-static-linux_amd64.tar.gz

That will just untar it into the relevant paths.

trapexit · Answer 16 · Fri Sep 08 2023 00:08:26 GMT+0800 (China Standard Time)

Or even simplier

VER=2.36.0
wget -qO- https://github.com/trapexit/mergerfs/releases/download/${VER}/mergerfs-static-linux_amd64.tar.gz | \
tar xvz -C /

CurlyFlow · Answer 17 · Fri Sep 08 2023 00:08:35 GMT+0800 (China Standard Time)

Thanks for your answer, i know it must be frustrating.

But just to be sure, if i saw it correct, when i unzip that file it creates somethign like usr/bin .. and as far as i know, it shoudl go directly to /bin (atleast that what i was before) - is that a problem?

Can i buy u a coffee?

trapexit · Answer 18 · Fri Sep 08 2023 00:11:27 GMT+0800 (China Standard Time)

I can't speak for how unraid sets things up. I've never used it. But typically /bin, /sbin, /usr/bin, /usr/sbin, /usr/local/{sbin,bin}/ and /opt/ are all common locations for software to be installed and looked for. Particular /usr/local for random, manually installed software. Hence why I use that location.

Can i buy u a coffee?

Thank you. Much appreciated.

https://github.com/trapexit/support

CurlyFlow · Answer 19 · Fri Sep 08 2023 00:17:28 GMT+0800 (China Standard Time)

https://github.com/trapexit/support Done. Dont have much, hope that helps.

Thats the old script:
mv /mnt/user/appdata/other/mergerfs/mergerfs /bin

So my guess would be it should be in /bin maybe has somethign to do with slackware and root or something? Or does it not care if its run in usr local instead of bin?

Edit: after some googling and chatgpd asking i guess it would work in both direcftories, will test it with next restart. :-)

Many thanks!

pfpulux · Answer 20 · Sun Oct 08 2023 15:08:17 GMT+0800 (China Standard Time)

have the similar error also an release 2.37.1 see musl-build.log, build with the voidlinux build system ./xbps-src -a x86_64-musl pkg mergerfs