Hang when launched from GNOME
ishitatsuyuki opened this issue · comments
Current Behavior
Hangs in black screen.
Backtrace:
(gdb) where
#0 0x00007f25da8e65e6 in epoll_pwait () at /usr/lib/libc.so.6
#1 0x000055e83892b9de in NativeMessagingHost::newNativeMessage() ()
#2 0x00007f25db53ac16 in QMetaObject::activate(QObject*, int, int, void**) ()
at /usr/lib/libQt5Core.so.5
#3 0x00007f25db547539 in QSocketNotifier::activated(int, QSocketNotifier::QPrivateSignal) () at /usr/lib/libQt5Core.so.5
#4 0x00007f25db547914 in QSocketNotifier::event(QEvent*) ()
at /usr/lib/libQt5Core.so.5
#5 0x00007f25dc9e5f2c in QApplicationPrivate::notify_helper(QObject*, QEvent*) () at /usr/lib/libQt5Widgets.so.5
#6 0x00007f25dc9eda06 in QApplication::notify(QObject*, QEvent*) ()
at /usr/lib/libQt5Widgets.so.5
#7 0x00007f25db50b060 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /usr/lib/libQt5Core.so.5
#8 0x00007f25db56696e in () at /usr/lib/libQt5Core.so.5
#9 0x00007f25d78dd270 in g_main_context_dispatch ()
at /usr/lib/libglib-2.0.so.0
#10 0x00007f25d78def69 in () at /usr/lib/libglib-2.0.so.0
#11 0x00007f25d78defae in g_main_context_iteration ()
at /usr/lib/libglib-2.0.so.0
#12 0x00007f25db565cd1 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /usr/lib/libQt5Core.so.5
#13 0x00007f25db50948b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>---Type <return> to continue, or q <return> to quit---
) () at /usr/lib/libQt5Core.so.5
#14 0x00007f25db5124d8 in QCoreApplication::exec() ()
at /usr/lib/libQt5Core.so.5
#15 0x000055e83884ff36 in main ()
It should be somewhat related to stdin. Anyway, I don't think you should use platform-specific polling in main thread: try QSocketNotifier. This is a weird problem, since not all fds can be polled (more weird on Windows).
Context
On launch by GNOME. Launching from the terminal doesn't trigger this issue.
Debug Info
Revision: 6409293
Operating system: Arch Linux
CPU architecture:x86_64
Kernel: 4.14.0-2-ARCH
Actually you have to use platform-specific polling with QSocketNotifier because otherwise the CPU consumption is going to hit the ceiling with constant signals to QSocketNotifier::activated(int). Windows doesn't use this at all because stdin cannot be used with QSocketNotifier with the OS.
I assume you are using a proxy and the wait gets stuck in the background? Maybe it would be smarter to disable whole QSocketNotifier when proxy is used and just listen for Unix Domain socket packets.
I think polling is too tricky to use here, and we should use an IO thread to achieve consistent behavior a/o not reinventing the wheel.
I'm using the proxy for sure, but I'm not sure what you mean with "stuck in background". The UI is black and completely unresponsive. Is it confusing the execution as native messaging launch?
Cross-platform IO thread is a tricky thing to do. You are welcome to try it :)
I mean is the UI black and unresponsive because of the epoll_wait or might there be another reason? Those signals shouldn't affect to UI.
I obtained the default stack trace; maybe it's not this thread. Let me get one for all threads next time.
I looked at the trace again, and the polling is definitely the cause of hang; main() is under the backtrace and it's definitely the main thread.
Are you able to build the sources by yourself? You could try if it makes a difference to change to epoll_wait timeout from -1 to for example 5000 milliseconds. I cannot reproduce the problem, but I think waiting for infinite here is definitely not the right way to do the polling :)
After your commit it can work after 5 seconds of wait. It is broken anyway.
We must not use any polling for stdin/stdout. The only portable way is spawn two threads for asynchronous read and write. It's malfunctioning on closed stdin, try with keepassxc < /dev/null.
This is really strange because that definitely shouldn't block anything.. and I cannot reproduce it.
The only portable way was the implementation I had few releases back: to use Boost with async functions. It worked like a charm but the KeePassXC devs were not so happy to add another dependency to KeePassXC so I needed to find a Qt based solution. AFAIK there's no other way to do this async with Qt than signals/slots and to use those with polling.
Do you mean you can't reproduce with stdin as /dev/null?
As I said, using a thread and doing synchronous IO there is the best solution. Also Qt has thread-safe signal facilities, so it should be easy. What's wrong with creating threads?
I can reproduce the /dev/null thing yes.
Actually I use synchronous IO under Windows already because there are no asynchronous methods for reading stdin. Of course I could test using the same thing under Linux and macOS but I'd still prefer to use as much asynchronous methods as possible.
It's not possible, because no platform allow you to poll everything. There are always pitfalls that you can't detect, and as a result poll() can return false positive, epoll() can hang.
And also reading from stdin can hang if it doesn't exist or there's no data. Terminating those synchronous IO threads by force is not very practical.
System calls can be interrupted with signals. I'm not sure if they are available in Qt/std though.
Also see the Windows way. This means we still need some #ifdef though.
Maybe it's possible. I need to do a lot more testing.
The problem with synchronous IO is that we'd still need different functions for reading under Windows and non-Windows platforms. Personally I would still prefer the Boost method.
I made a new branch sync for testing the issue. Here's no async, just a QThread with the synchronous methods. If you use it, you can clearly see the problems it's having. Even when the thread is signaled to stop it might have get stuck in reading stdin and won't quit. Using QThread::terminate() is not the solution.
You are free to test other alternatives :)
I have come up with two methods, both requires platform-specific calls.
The easy way:
Duplicate stdin, terminate the IO with close.
The hard way:
Get the thread's native handle (by either migrating to std::thread or using Qt's private headers), and call CancelIoEx or pthread_kill to interrupt the IO. Standard library IO functions must not be used as there's a possibility that it will ignore the interruption and retry.
Duplicating the stdin might work. I'll test it out. pthread_kill won't because terminating or killing the thread by force is not the right solution.
No, I didn't mean that way. pthread_kill is a way to signal thread, not necessarily forceful termination. IIRC any non terminating signal can trigger an EINTR.
I updated the branch with the stdin duplicating. Seems to work ok with Linux, but it needs some testing under other operating systems.
After trying several things with QThread and different kind of stdin reading I'm still in favour of the current implementation which is:
- Linux/Unix/macOS: using asynchronous QSocketNotifier and platform-specific polling with
stdin - Windows: using synchronous calls for reading
stdin
If you have a possibility to do async, do async.
Can you please check does Gnome has any command line arguments or stuff that could cause this issue? Using dmenu/rofl etc doesn't have this kind of behaviour.
It seems it's working on GNOME so far. We don't really need to consider edge cases here (such as file as stdin), so I think this implementation is fine. Close when you have tested on all platforms and merged the branch.
To be clear: I'm not going to merge this testing branch to the PR.
The problem is that epoll can fail (/dev/null is not pollable). The issue is here (obtained with strace):
epoll_ctl(42, EPOLL_CTL_ADD, 0, 0x7fff4008025c) = -1 EPERM (Operation not permitted)
epoll_pwait(42, <hang...>
You are correct. There was an return value check missing. Fixed it and now it seems to work.
有り難う :)