Segfault when stress-testing an http server
MagnificentPako opened this issue · comments
I tried to write a multi-threaded-ish http server for emojicode. After playing around with the language for some time I actually got a working implementation (which even has a simplistic router so one could write full REST applications in emojicode) which can be found here: https://git.nightmares.pro/EmojiChat/http_server .
It never failed on me when doing simple one-connection tests. Then I tried to benchmark it with wrk (arguments used were -c400 -t8 -d10s resulting in 400 connections, distributed on 8 threads for 10 seconds) which causes a segfault to happen. I assume this should be reproducable when cloning the repo, then compiling and running main.emojic. Now you can do wrk -c400 -t8 -d10s http://localhost:8080/test which should result in the server giving you a segfault.
This is what I get when doing coredumpctl gdb https://gist.github.com/MagnificentPako/b460c67dae92dbdf5b4d2de570889e50
Greetings,
Paul
#Edit:
Because the git is down right now here's an up-to-date link: https://git.procrastination.life/Pako/http_server
Thanks for reporting. I could reproduce the problem and there seems to be some kind of memory bug involved. I'll look into this.
I also encountered a similar bug (or maybe the same bug?) - a program that should work in O(n) memory (I'd estimate it should take <1 MB) segfaulted the virtual machine for n > 7 unless the heap size was increased. You can see it here. It's a program that solves this challenge involving maths and stamps. I tried to reduce this to a simpler example, but it doesn't seem to work. BTW, fixing this can make the gc stress tests pass. This program also allowed me to find #87.
@NieDzejkob Thanks for reporting. It would be helpful to report issues like these immediately instead of just commenting them on SE.
Emojicode performs only one allocation for the heap, which is properly guarded against a null pointer, so that's definitely not the issue. If this is actually related to this bug and #56, I would expect it to feature multithreading, which seems to be the root cause of the other bugs. How did you conclude that "fixing this can make the gc stress tests pass"?
@NieDzejkob At first glance your bug seems different to the others, but I'll have to look at it further.
Your program does not allocate memory linearly because closures always allocate memory as of now. (An optimization for the future could be implemented here.)
I didn't report it immediately because I wanted to get a simpler example of failing code, then I got distracted, etc. This issue reminded me of that. Now that I think about it, should this be a separate issue?
@NieDzejkob You were indeed right. Your program was affected by a somewhat obvious bug that I fixed yesterday, but now it shows the symptoms of #56. I will investigate further, before we decide how to categorize or separate this issue from what this thread here is actually about.
I released https://github.com/emojicode/emojicode/releases/tag/v0.5.3 which resolves the bug that affected @NieDzejkob's program and probably also the HTTP Server example.
The GC stress test, however, is still affected by a rare bug that seems strongly related to threading. The HTTP Server code terminates with 🚨 Fatal Error: Unexpectedly found ✨ while unwrapping a 🍬. on stress testing now, which could be related to the same bug.
I'll keep looking into this.
🚨 Fatal Error: Unexpectedly found ✨ while unwrapping a 🍬. is caused by line 62, http_server.emojic (in the version initially provided) and seems legit. There are probably just not as many system resources available as would be required.
There was one additional memory bug, which should be fixed now and will be released in a new version shortly.