pluginval intermittently crashes with SIGPIPE
bepzi opened this issue · comments
This is from a pluginval build from the develop branch.
Trying to test a "Hello World" VST2 plugin on Arch Linux. When running pluginval in Debug mode, it will sometimes crash with "SIGPIPE, Broken pipe". This is a Rust project, it's pretty much just the example plugin from the vst-rs README.
EDIT: The fact that it's Rust-based doesn't seem to matter. I was able to recreate this with a standard C++ JUCE-based VST2 plugin.
When running in Release mode, it crashes maybe 9/10 times. When running in debug mode, it crashes maybe 1 in 10 times.
(gdb) r
Starting program: /home/z/downloads/pluginval/bin/linux/pluginval --validate /home/z/development/rust/sturdy/target/debug/libsturdy_plugin.so
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
JUCE v5.4.7
Random seed: 0x7c85b03
-----------------------------------------------------------------
Starting test: CommandLineTests / Merge environment variables...
All tests completed successfully
-----------------------------------------------------------------
Starting test: CommandLineTests / Command line defaults...
All tests completed successfully
-----------------------------------------------------------------
Starting test: CommandLineTests / Command line parser...
All tests completed successfully
-----------------------------------------------------------------
Starting test: CommandLineTests / Command line random...
All tests completed successfully
-----------------------------------------------------------------
Starting test: AllocatorInterceptorTests / Ensure separate allocators are used for threads...
[New Thread 0x7ffff6c1a700 (LWP 16671)]
All tests completed successfully
[Thread 0x7ffff6c1a700 (LWP 16671) exited]
-----------------------------------------------------------------
Starting test: AllocatorInterceptorTests / Ensure all allocations pass...
All tests completed successfully
-----------------------------------------------------------------
Starting test: AllocatorInterceptorTests / Ensure all allocations fail on MessageThread...
All tests completed successfully
-----------------------------------------------------------------
Starting test: AllocatorInterceptorTests / Ensure allocation violation status is reset...
All tests completed successfully
-----------------------------------------------------------------
Starting test: AllocatorInterceptorTests / Ensure all allocations pass on MessageThread, fail on background thread...
[New Thread 0x7ffff6c1a700 (LWP 16672)]
[Thread 0x7ffff6c1a700 (LWP 16672) exited]
All tests completed successfully
-----------------------------------------------------------------
Starting test: AllocatorInterceptorTests / Ensure array allocations are caught...
All tests completed successfully
-----------------------------------------------------------------
Starting test: AllocatorInterceptorTests / Ensure allocations are thrown...
All tests completed successfully
[Detaching after fork from child process 16673]
[New Thread 0x7ffff6c1a700 (LWP 16674)]
[New Thread 0x7ffff6419700 (LWP 16675)]
pluginval v0.2.6 - JUCE v5.4.7
Thread 5 "IPC ping" received signal SIGPIPE, Broken pipe.
[Switching to Thread 0x7ffff6419700 (LWP 16675)]
0x00007ffff7bc97df in write () from /usr/lib/libpthread.so.0
(gdb) bt
#0 0x00007ffff7bc97df in write () from /usr/lib/libpthread.so.0
#1 0x000055555576b8c3 in juce::NamedPipe::Pimpl::write (timeOutMilliseconds=<optimized out>, numBytesToWrite=16, sourceBuffer=0x7fffe8000ba0 "\004\257+q\b", this=0x555555ded2d0)
at ../../modules/juce/modules/juce_core/native/juce_posix_NamedPipe.cpp:96
#2 juce::NamedPipe::write (this=0x555555dee410, sourceBuffer=sourceBuffer@entry=0x7fffe8000ba0, numBytesToWrite=numBytesToWrite@entry=16, timeOutMilliseconds=<optimized out>)
at ../../modules/juce/modules/juce_core/native/juce_posix_NamedPipe.cpp:235
#3 0x00005555557b40e0 in juce::InterprocessConnection::writeData (this=this@entry=0x555555df0600, data=0x7fffe8000ba0, dataSize=16)
at ../../modules/juce/modules/juce_events/interprocess/juce_InterprocessConnection.cpp:173
#4 0x00005555557b417a in juce::InterprocessConnection::sendMessage (this=0x555555df0600, message=...) at ../../modules/juce/modules/juce_core/memory/juce_HeapBlock.h:200
#5 0x00005555557b464a in juce::ChildProcessMaster::sendMessageToSlave (this=<optimized out>, mb=...) at ../../modules/juce/modules/juce_events/interprocess/juce_ConnectedChildProcess.cpp:135
#6 0x00005555557be75f in non-virtual thunk to juce::ChildProcessMaster::Connection::sendPingMessage(juce::MemoryBlock const&) ()
at ../../modules/juce/modules/juce_events/interprocess/juce_ConnectedChildProcess.cpp:106
#7 0x00005555557be7d0 in juce::ChildProcessPingThread::run (this=0x555555df0670) at ../../modules/juce/modules/juce_events/interprocess/juce_ConnectedChildProcess.cpp:67
#8 0x000055555576bde2 in juce::Thread::threadEntryPoint (this=0x555555df0670) at ../../modules/juce/modules/juce_core/threads/juce_Thread.cpp:96
#9 0x000055555576bf75 in juce::juce_threadEntryPoint (userData=<optimized out>) at ../../modules/juce/modules/juce_core/threads/juce_Thread.cpp:118
#10 0x000055555576bf83 in juce::threadEntryProc (userData=<optimized out>) at ../../modules/juce/modules/juce_core/native/juce_posix_SharedCode.h:838
#11 0x00007ffff7bbf422 in start_thread () from /usr/lib/libpthread.so.0
#12 0x00007ffff77b0b83 in clone () from /usr/lib/libc.so.6
(gdb)
That usually means that the plugin crashed and the pipe was broken.
Can you try testing with "validate-in-process" enabled and see exactly where it crashes?
"validate-in-process" gave two different results:
- The tests passed
- Clicking "Test Selected" caused pluginval to immediately crash on an assertion, specifically this JUCE code:
// modules/juce_events/interprocess/juce_ConnectedChildProcess.cpp
bool ChildProcessMaster::sendMessageToSlave (const MemoryBlock& mb)
{
if (connection != nullptr)
return connection->sendMessage (mb);
jassertfalse; // this can only be used when the connection is active!
return false;
}Running pluginval with "validate-in-process" in GDB also yielded the SIGPIPE again, same backtrace as before.
Sorry, I should have been a bit clearer, if you're using the UI, there's an option called "Validate in process", try ticking this and running it again. It shouldn't hit that sendMessageToSlave path at all then. (If it does, could you post the stack trace please?)
If you're using the command line version, you'll need to pass the "--validate-in-process" option.
My bad. I passed --validate-in-process and the GUI came up, but the option wasn't ticked.
Now I get this in my terminal emulator (the GUI crashed):
Starting test: pluginval / Open plugin (cold)...
JUCE Assertion failure in juce_Thread.cpp:233
!! killing thread by force !!
terminate called without an active exception
I ran it with GDB and after about 20-30 tries it crashed again, the backtrace is:
Starting test: pluginval / Open plugin (cold)...
JUCE Assertion failure in juce_Thread.cpp:233
--Type <RET> for more, q to quit, c to continue without paging--
Thread 1 "pluginval" received signal SIGTRAP, Trace/breakpoint trap.
0x00007ffff76ed6bb in kill () from /usr/lib/libc.so.6
(gdb) bt
#0 0x00007ffff76ed6bb in kill () from /usr/lib/libc.so.6
#1 0x00005555557348c7 in juce::Thread::stopThread (this=this@entry=0x555555f8bff8, timeOutMilliseconds=timeOutMilliseconds@entry=5000) at ../../modules/juce/modules/juce_core/threads/juce_Thread.cpp:233
#2 0x0000555555610f6f in ValidatorSlaveProcess::~ValidatorSlaveProcess (this=0x555555f8bfe0, __in_chrg=<optimized out>) at ../../Source/Validator.cpp:312
#3 ValidatorSlaveProcess::~ValidatorSlaveProcess (this=0x555555f8bfe0, __in_chrg=<optimized out>) at ../../Source/Validator.cpp:313
#4 0x000055555560e54f in std::default_delete<ValidatorSlaveProcess>::operator() (__ptr=<optimized out>, this=0x555555fb0b98) at /usr/include/c++/10.1.0/bits/unique_ptr.h:78
#5 std::unique_ptr<ValidatorSlaveProcess, std::default_delete<ValidatorSlaveProcess> >::~unique_ptr (this=0x555555fb0b98, __in_chrg=<optimized out>) at /usr/include/c++/10.1.0/bits/unique_ptr.h:360
#6 ValidatorMasterProcess::~ValidatorMasterProcess (this=0x555555fb0a50, __in_chrg=<optimized out>) at ../../Source/Validator.cpp:543
#7 0x000055555560e703 in ValidatorMasterProcess::~ValidatorMasterProcess (this=0x555555fb0a50, __in_chrg=<optimized out>) at ../../Source/Validator.cpp:543
#8 0x000055555560b525 in std::default_delete<ValidatorMasterProcess>::operator() (__ptr=<optimized out>, this=0x555555decd58) at /usr/include/c++/10.1.0/bits/unique_ptr.h:78
#9 std::__uniq_ptr_impl<ValidatorMasterProcess, std::default_delete<ValidatorMasterProcess> >::reset (__p=0x0, this=0x555555decd58) at /usr/include/c++/10.1.0/bits/unique_ptr.h:181
#10 std::unique_ptr<ValidatorMasterProcess, std::default_delete<ValidatorMasterProcess> >::reset (__p=0x0, this=0x555555decd58) at /usr/include/c++/10.1.0/bits/unique_ptr.h:455
#11 Validator::handleAsyncUpdate (this=0x555555decd00) at ../../Source/Validator.cpp:773
#12 0x00005555557bb426 in juce::AsyncUpdater::AsyncUpdaterMessage::messageCallback (this=<optimized out>) at ../../modules/juce/modules/juce_events/broadcasters/juce_AsyncUpdater.cpp:34
#13 0x00005555557c2279 in juce::InternalMessageQueue::InternalMessageQueue()::{lambda(int)#1}::operator()(int) const (__closure=0x555555dec968, fd=4)
at ../../modules/juce/modules/juce_core/memory/juce_ReferenceCountedObject.h:392
#14 0x00005555557c2362 in std::__invoke_impl<void, juce::InternalMessageQueue::InternalMessageQueue()::{lambda(int)#1}&, int>(std::__invoke_other, juce::InternalMessageQueue::InternalMessageQueue()::{lambda(int)#1}&, int&&) (__f=...) at /usr/include/c++/10.1.0/bits/invoke.h:148
#15 std::__invoke_r<void, juce::InternalMessageQueue::InternalMessageQueue()::{lambda(int)#1}&, int>(std::__is_invocable&&, (juce::InternalMessageQueue::InternalMessageQueue()::{lambda(int)#1}&)...) (__fn=...)
at /usr/include/c++/10.1.0/bits/invoke.h:153
#16 std::_Function_handler<void (int), juce::InternalMessageQueue::InternalMessageQueue()::{lambda(int)#1}>::_M_invoke(std::_Any_data const&, int&&) (__functor=..., __args#0=<optimized out>)
at /usr/include/c++/10.1.0/bits/std_function.h:291
#17 0x00005555557c1147 in std::function<void (int)>::operator()(int) const (this=this@entry=0x555555dec968, __args#0=<optimized out>, __args#0@entry=4) at /usr/include/c++/10.1.0/bits/std_function.h:248
#18 0x00005555557c11f7 in juce::InternalRunLoop::dispatchPendingEvents (this=this@entry=0x555555dec8e0) at ../../modules/juce/modules/juce_events/native/juce_linux_Messaging.cpp:186
#19 0x00005555557ba41a in juce::MessageManager::dispatchNextMessageOnSystemQueue (returnIfNoPendingMessages=returnIfNoPendingMessages@entry=false)
at ../../modules/juce/modules/juce_events/native/juce_linux_Messaging.cpp:294
#20 0x00005555557ba4fe in juce::MessageManager::runDispatchLoop (this=0x555555dec870) at ../../modules/juce/modules/juce_events/messages/juce_MessageManager.cpp:128
#21 0x00005555557bb378 in juce::JUCEApplicationBase::main () at ../../modules/juce/modules/juce_events/messages/juce_ApplicationBase.cpp:262
#22 0x00005555557bb3c6 in juce::JUCEApplicationBase::main (argc=<optimized out>, argv=<optimized out>) at ../../modules/juce/modules/juce_events/messages/juce_ApplicationBase.cpp:240
#23 0x00005555555e8e68 in main (argc=<optimized out>, argv=<optimized out>) at ../../Source/Main.cpp:183
That code is:
bool Thread::stopThread (const int timeOutMilliseconds)
{
// agh! You can't stop the thread that's calling this method! How on earth
// would that work??
jassert (getCurrentThreadId() != getThreadId());
const ScopedLock sl (startStopLock);
if (isThreadRunning())
{
signalThreadShouldExit();
notify();
if (timeOutMilliseconds != 0)
waitForThreadToExit (timeOutMilliseconds);
if (isThreadRunning())
{
// very bad karma if this point is reached, as there are bound to be
// locks and events left in silly states when a thread is killed by force..
jassertfalse; // <----------- CRASHES HERE
Logger::writeToLog ("!! killing thread by force !!");
killThread();
threadHandle = nullptr;
threadId = {};
return false;
}
}
return true;
}I think you need to print out the stack traces from the other running threads. This is indicating that the main thread is trying to delete a background thread (the ValidatorSlaveProcess) whilst it is still running. You need to view the stack trace of that thread to see what's blocking it.
I think this should be fixed with 1.0.0 as it doesn't use this pipe mechanism any more