SIGSEGV on Alpine 3.18.2 with webtorrent@2.1.11 (due to utp-native@2.5.3)
proton-ab opened this issue · comments
What version of this package are you using?
v2.1.11
What operating system, Node.js, and npm version?
node:18-alpine@bf6c61feabc1a1bd565065016abe77fa378500ec75efa67f5b04e5e5c4d447cd (sha256:d5b2a7869a4016b1847986ea52098fa404421e44281bb7615a9e3615e07f37fb)
What happened?
SIGSEGV due to utp-native (see logs)
root@michiru:/opt/isla# docker compose run fetcher sh
/app # apk add gdb
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
(1/18) Installing libexpat (2.5.0-r1)
(2/18) Installing gmp (6.2.1-r3)
(3/18) Installing ncurses-terminfo-base (6.4_p20230506-r0)
(4/18) Installing libncursesw (6.4_p20230506-r0)
(5/18) Installing libbz2 (1.0.8-r5)
(6/18) Installing libffi (3.4.4-r2)
(7/18) Installing gdbm (1.23-r1)
(8/18) Installing xz-libs (5.4.3-r0)
(9/18) Installing mpdecimal (2.5.1-r2)
(10/18) Installing libpanelw (6.4_p20230506-r0)
(11/18) Installing readline (8.2.1-r1)
(12/18) Installing sqlite-libs (3.41.2-r2)
(13/18) Installing python3 (3.11.4-r0)
(14/18) Installing python3-pycache-pyc0 (3.11.4-r0)
(15/18) Installing pyc (0.1-r0)
(16/18) Installing python3-pyc (3.11.4-r0)
(17/18) Installing zstd-libs (1.5.5-r4)
(18/18) Installing gdb (13.1-r9)
Executing busybox-1.36.1-r0.trigger
OK: 70 MiB in 46 packages
/app # gdb --args node dist/index.js
GNU gdb (GDB) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from node...
(gdb) run
Starting program: /usr/local/bin/node dist/index.js
[New LWP 18]
[New LWP 19]
[New LWP 20]
[New LWP 21]
[New LWP 22]
[New LWP 23]
[New LWP 24]
[New LWP 25]
[New LWP 26]
[New LWP 27]
(node:15) ExperimentalWarning: Import assertions are not a stable feature of the JavaScript language. Avoid relying on their current behavior and syntax as those might change in a future version of Node.js.
(Use `node --trace-warnings ...` to show where the warning was created)
(node:15) ExperimentalWarning: Importing JSON modules is an experimental feature and might change at any time
Environment variable LOG_LEVEL is not set and has been defaulted to debug
[Detaching after fork from child process 28]
[21:14:48] INFO main: Starting fetcher
[21:14:48] INFO WebServer: Webserver now listening on 0.0.0.0 over port 3004
[21:14:48] INFO IRCNetwork: Attempting to connect to IRC at irc.rizon.net:6667
[21:14:48] INFO IRCNetwork: Attempting to connect to IRC at irc.animebytes.tv:+7000
[21:14:48] WARN IRCNetwork: Unexpected NickServ notice on animebytes: This nickname is registered and protected. If it is your
[21:14:48] WARN IRCNetwork: Unexpected NickServ notice on animebytes: nick, type /msg NickServ IDENTIFY password. Otherwise,
[21:14:48] WARN IRCNetwork: Unexpected NickServ notice on animebytes: please choose a different nick.
[New LWP 29]
[21:14:48] INFO IRCNetwork: Successfully connected and identified to IRC server animebytes
[21:14:50] INFO IRCNetwork: Successfully connected to IRC server rizon
[21:14:50] INFO IRCNetwork: Joined #isla on animebytes
[21:14:50] DEBUG Reloader: Starting shows definition reload
[21:14:50] INFO EpisodeModel: Starting fetch for Insomniacs after school - 12 [720p][SubsPlease][39DB60F0].mkv
[21:14:50] INFO ABClient: Logging into AnimeBytes with user Isla
[21:14:51] INFO Reloader: Shows definition changed; updating now
[21:14:51] DEBUG Reloader: Shows definition reload complete
[21:14:51] DEBUG Reloader: Starting sources refresh
Thread 1 "node" received signal SIGSEGV, Segmentation fault.
0x0000000000004fb6 in ?? ()
(gdb) bt
#0 0x0000000000004fb6 in ?? ()
#1 0x00007ffd72a0e49a in utp_writev () from /app/node_modules/utp-native/prebuilds/linux-x64/node.napi.node
#2 0x00007ffd72a0ba66 in utp_napi_connection_write(napi_env__*, napi_callback_info__*) () from /app/node_modules/utp-native/prebuilds/linux-x64/node.napi.node
#3 0x0000555555e34d1e in v8impl::(anonymous namespace)::FunctionCallbackWrapper::Invoke(v8::FunctionCallbackInfo<v8::Value> const&) ()
#4 0x0000555556103e08 in v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) ()
#5 0x000055555610482f in v8::internal::MaybeHandle<v8::internal::Object> v8::internal::(anonymous namespace)::HandleApiCallHelper<false>(v8::internal::Isolate*, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::HeapObject>, v8::internal::Handle<v8::internal::FunctionTemplateInfo>, v8::internal::Handle<v8::internal::Object>, v8::internal::BuiltinArguments) ()
#6 0x0000555556104d17 in v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) ()
#7 0x0000555556b248b9 in Builtins_CEntry_Return1_DontSaveFPRegs_ArgvOnStack_BuiltinExit ()
#8 0x0000555556aa8810 in Builtins_InterpreterEntryTrampoline ()
#9 0x0000156b8e0015b9 in ?? ()
#10 0x00003718699e6ba1 in ?? ()
#11 0x0000000700000000 in ?? ()
#12 0x0000156b8e001689 in ?? ()
#13 0x0000256e754aee09 in ?? ()
#14 0x000034c86de6b9d9 in ?? ()
#15 0x000037f03e5b1051 in ?? ()
#16 0x000034c86de6b9d9 in ?? ()
#17 0x0000256e754aee09 in ?? ()
#18 0x00003718699e6ba1 in ?? ()
#19 0x0000005400000000 in ?? ()
#20 0x000016f70c7f22c9 in ?? ()
#21 0x0000000000000004 in ?? ()
#22 0x00003718699ff1e9 in ?? ()
#23 0x0000256e75491701 in ?? ()
#24 0x00007fffffffa968 in ?? ()
#25 0x0000555556aa8810 in Builtins_InterpreterEntryTrampoline ()
#26 0x000034c86de6b501 in ?? ()
#27 0x000037f03e5b1051 in ?? ()
#28 0x0000156b8e004861 in ?? ()
#29 0x000016f70c7dc6b9 in ?? ()
#30 0x000016f70c7dc6b9 in ?? ()
#31 0x0000156b8e004861 in ?? ()
#32 0x000037f03e5b1051 in ?? ()
#33 0x000034c86de6b501 in ?? ()
#34 0x00003718699ff1e9 in ?? ()
#35 0x0000009600000000 in ?? ()
#36 0x000016f70c7f2119 in ?? ()
#37 0x0000000000000008 in ?? ()
#38 0x0000256e75497d49 in ?? ()
#39 0x0000256e75497969 in ?? ()
Interestingly, this used to work with node:18-alpine@a3f2350bd3eb48525f801b57934300c11aa3610086b708854ab1c1045c018519 (sha256:47d97b93629d9461d64197773966cc49081cf4463b1b07de5a38b6bd5acfbe9d) which was based on Alpine 3.17.3, however at the time we were using webtorrent@2.0.16. Either something in how webtorrent interacts with utp-native has changed or something between Alpine 3.17.3 and Alpine 3.18.2 broke it. For now, passing utp: false to WebTorrent instance gets rid of it.
What did you expect to happen?
Program runs normally.
Are you willing to submit a pull request to fix this bug?
Not qualified.
Just tested with current node:18-alpine3.17 and it shows same issue, so short of some change in Alpine that got backported to 3.17 it points at webtorrent.
before I start helping you out: unless you're an admin or have EXPLICIT permission, using webtorrent will DEFINETLY get you banned on AB, as to my knowledge WT isn't a whitelisted client, if you have permission then:
- this is an issue with a dependency, mainly utp-native, as I don't know the dependencies code I might not be of much help
- I'd need more detailed information on what actually happens, try running webtorrent with an env variable
DEBUG=*, tho be mindful that the logs it generates will be enormous, and if you're using some sort of debugger like chromium remote debugger, it will quickly run out of memory - what are the reproduction steps, what are you trying to do?
- does webtorrent flat out crash and exit the entire process?
- a stack trace of the JS not only C would be good
if the webtorrent process actually exits upon eval, then we have a regression, most likely caused by my changes, which is really bad [edit: that would mean the process crashes when utp is disabled which you said it doesnt], if it exits when connecting to a peer, then it's an error that needs to be handled, if the process just flat out crashes and doesn't exit gracefully, then utp-native needs to fix this issue, which is unlikely as mafintosh isn't really that interested in webtorrent anymore
you mention this worked, but doesn't, did upgrading node break it, or did upgrading webtorrent break it
Hi, you can find what we use webtorrent for on https://github.com/anniemaybytes/fetcher.
To answer your questions:
- Yes, this is issue with
utp-native - I'll try running it with
DEBUGwhen I have a moment - For the purpose of causing issue, I believe just instantiating
webtorrentwithutp: trueand having it try to download a torrent that has UTP peers would trigger it. - Yes, this is segmentation fault, entire process is killed.
- There is no JS trace as this is SIGSEGV, process is killed immediately by kernel with no real chance for JavaScript to do much.
if the process just flat out crashes and doesn't exit gracefully, then utp-native needs to fix this issue, which is unlikely as mafintosh isn't really that interested in webtorrent anymore
utp is enabled by default in webtorrent. If the issue will not be resolved upstream then I believe webtorrent needs to consider disabling it or dropping utp-native entirely in favor of some pure JavaScript implementation.
you mention this worked, but doesn't, did upgrading node break it, or did upgrading webtorrent break it
Will attempt to trace it soon.
I tried reaching you on IRC but it didn't really work out on my end as I don't have an IRC client set up, and I'm quite indisposed at the moment, I'll try to resolve it to try to help you debug it, as this doesn't seem like a webtorrent issue as I'm running utp native just fine in my apps on latest versions of webtorrent, and more a docker/Linux issue
a good place to start would be not using prebuilt dists as you are right now
Looks like the fact that previous build was working was due to a bug fixed in #2561. Seems like this has been broken all along on Alpine and I've just discovered it by accident because I've chosen to enable UTP support (it was disabled explicitly before) when it was broken and non-functioning.
yeah ever since v2 UTP was broken and was fixed in 2.0.29 as you said, I've had a look at your code, and you're not doing anything wrong, so i would simply recommend running npm rebuild in the docker container before build
Yes, it would seem that rebuilding utp-native does fix the issue. It might be prudent for webtorrent to document this somewhere, as Alpine-based containers are pretty popular.
welp this was tracked down as a utp-native issue, tho it can't be fully blamed on them, TLDR a prebuild was used which wasn't compatble with alpine, and simply running rebuild fixed this
I don't think we'll document this as it's one of those "you use linux, you should know" and/or "we won't check compatibility for each version of each linux distro out there" issues, which are both a deep rabbit hole