Twinkle crashes with PCM error message

Question

Twinkle crashes with PCM error message

urbanware-org opened this issue a year ago · comments

I'm using Twinkle on Fedora 37 and Twinkle 1.10.3 with PipeWire as follows:

Actually this works fine, however, once in a while, Twinkle crashes "silently". The tray icon suddenly disappears without any error message. After that happened a couple of times, I ran Twinkle on a shell to see what happens when it crashes again. Every time it crashes, it does with this error message:

twinkle: pcm.c:3380: snd_pcm_areas_copy: Assertion `dst_areas' failed.

This occurs in the following two situations:

Incoming calls: I don't hear the ringtone or see the usual person calling dialog. Instead Twinkle instantly crashes with the message mentioned above.
Outgoing calls: After I dial the number and confirm the number to start the call. Instead of getting the dial tone, Twinkle also crashes the same way.

This happens at irregular intervals. Sometimes it doesn't happen at all in one day, sometimes several times. So, I'm unable to manually reproduce it in any way.

If I'm not mistaken, the error message comes from ALSA libraries. If relevant, this is the list of the installed ALSA packages:

alsa-lib-1.2.9-1.fc37.x86_64
alsa-sof-firmware-2.2.5-1.fc37.noarch
alsa-ucm-1.2.9-1.fc37.noarch
alsa-utils-1.2.9-1.fc37.x86_64

While I'm on the phone, I don't have any problems at all.

cyberbeat · Answer 1 · Wed Oct 04 2023 23:27:22 GMT+0800 (China Standard Time)

I use same configuration (pipewire), and have similar "silent" crashes.

Frédéric Brière · Answer 2 · Sat Oct 07 2023 09:12:50 GMT+0800 (China Standard Time)

@urbanware-org Thank you very much for taking the time to provide so many details; this was quite helpful and very much appreciated!

I think I know where the (ALSA) bug is located, but just to make sure, could you provide us with a backtrace? We have instructions on how to generate one on our Wiki; Fedora also has its own information on the subject. In addition:

Please make sure to install the alsa-lib-debuginfo package.
Feel free to issue a simple backtrace (instead of thread apply all backtrace) to keep things simple, since I'm not interested in the other threads.

Thanks in advance!

Frédéric Brière · Answer 3 · Sat Oct 07 2023 09:32:24 GMT+0800 (China Standard Time)

For the record, here's where I think the bug is located (with another copy in snd_pcm_mmap_read_areas()):

https://github.com/alsa-project/alsa-lib/blob/81a7a93636d9472fcb0c2ff32d9bfdf6ed10763d/src/pcm/pcm_mmap.c#L86-L90

__snd_pcm_mmap_begin() is called to set up pcm_areas (on the stack), but no check is performed on its return value. If the call failed, pcm_areas is not set and contains garbage, and any future attempt to use it as source/destination is likely to cause an error. (Fortunately, it happened to contain 0 and is detected by an assertion, instead of any random value that would trigger a mysterious segfault later on.)

After some experimentation, it appears that the value assigned to pcm_areas remains the same from one call to the next, and also happens to survive on the stack. Basically, once the first call to __snd_pcm_mmap_begin() has been successful, any subsequent failures will not have any impact, hence why all crashes occur at the very start of any audio playback. (It's not that errors no longer occur, but rather that we're blissfully unaware of them.)

Frédéric Brière · Answer 4 · Sat Oct 07 2023 09:35:44 GMT+0800 (China Standard Time)

@cyberbeat Do you get the same "failed assertion" message? If so, would you be able to generate a backtrace as I have explained in my previous comment? Thanks!

Ralf Kilian · Answer 5 · Sat Oct 07 2023 23:00:31 GMT+0800 (China Standard Time)

Thank you very much for taking the time to provide so many details; this was quite helpful and very much appreciated!

@fbriere You're welcome, I'm always glad to help!

I think I know where the (ALSA) bug is located, but just to make sure, could you provide us with a backtrace?

I'm currently not in office having a few days off, but I will be back at work on October, 16^th and can create a backtrace then.

cyberbeat · Answer 6 · Wed Oct 11 2023 15:51:17 GMT+0800 (China Standard Time)

@cyberbeat Do you get the same "failed assertion" message? If so, would you be able to generate a backtrace as I have explained in my previous comment? Thanks!

Yes:

Language name: "de"
warning: The VAD has been replaced by a hack pending a complete rewrite
warning: The VAD has been replaced by a hack pending a complete rewrite
warning: discarded first playback frame
warning: The VAD has been replaced by a hack pending a complete rewrite
warning: The VAD has been replaced by a hack pending a complete rewrite
warning: discarded first playback frame
warning: The VAD has been replaced by a hack pending a complete rewrite
warning: discarded first playback frame
warning: The VAD has been replaced by a hack pending a complete rewrite
warning: discarded first playback frame
twinkle: pcm.c:3340: snd_pcm_areas_copy: Zusicherung »dst_areas« nicht erfüllt.
Abgebrochen (Speicherabzug geschrieben)

unluckily, I cannot find a core dump (although started from shell with ulimit -c unlimited, perhaps because of
/proc/sys/kernel/core_pattern = |/bin/false

cyberbeat · Answer 7 · Wed Oct 11 2023 17:12:30 GMT+0800 (China Standard Time)

Finally a backtrace:

Thread 1 (Thread 0x7fd3f5ffb700 (LWP 24765)):
#0  0x00007fd429f57c6b in raise () from /lib64/libc.so.6
#1  0x00007fd429f59305 in abort () from /lib64/libc.so.6
#2  0x00007fd429f4fc6a in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007fd429f4fcf2 in __assert_fail () from /lib64/libc.so.6
#4  0x00007fd42cdae352 in snd_pcm_areas_copy (dst_areas=dst_areas@entry=0x7fd3f5ffa8e0, dst_offset=dst_offset@entry=0, src_areas=src_areas@entry=0x0, src_offset=src_offset@entry=2048, channels=channels@entry=1, frames=frames@entry=342, format=SND_PCM_FORMAT_S16_LE) at pcm.c:3341
#5  0x00007fd42cdae3ff in snd_pcm_areas_copy_wrap (dst_channels=0x7fd3f5ffa8e0, dst_offset=0, dst_size=342, src_channels=0x0, src_offset=2048, src_size=4096, channels=1, frames=342, format=SND_PCM_FORMAT_S16_LE) at pcm.c:3432
#6  0x00007fd405b51f89 in ?? () from /usr/lib64/alsa-lib/libasound_module_pcm_pipewire.so
#7  0x00007fd405b521e4 in ?? () from /usr/lib64/alsa-lib/libasound_module_pcm_pipewire.so
#8  0x00007fd404381ccd in ?? () from /usr/lib64/libpipewire-0.3.so.0
#9  0x00007fd3fc543653 in ?? () from /usr/lib64/spa-0.2/audioconvert/libspa-audioconvert.so
#10 0x00007fd40435b689 in ?? () from /usr/lib64/libpipewire-0.3.so.0
#11 0x00007fd40440800e in ?? () from /usr/lib64/spa-0.2/support/libspa-support.so
#12 0x00007fd404332380 in ?? () from /usr/lib64/libpipewire-0.3.so.0
#13 0x00007fd42ea8e6ea in start_thread () from /lib64/libpthread.so.0
#14 0x00007fd42a02494f in clone () from /lib64/libc.so.6

Using

Name        : libasound2
Version     : 1.2.6.1
Release     : 150400.1.4
Architecture: x86_64
Install Date: Mi 26 Apr 2023 13:02:07 CEST
Group       : System/Libraries
Size        : 1211543
License     : LGPL-2.1-or-later
Signature   : RSA/SHA256, Sa 07 Mai 2022 23:41:20 CEST, Key ID 70af9e8139db7c82
Source RPM  : alsa-1.2.6.1-150400.1.4.src.rpm
Build Date  : Sa 07 Mai 2022 23:38:28 CEST
Build Host  : sheep64
Relocations : (not relocatable)
Packager    : https://www.suse.com/
Vendor      : SUSE LLC <https://www.suse.com/>
URL         : https://www.alsa-project.org
Summary     : Advanced Linux Sound Architecture Library
Description :
This package contains the library for ALSA, Advanced Linux Sound
Architecture.
Distribution: SUSE Linux Enterprise 15

Btw most times I have this crash at the end of calls, and I often don't recognize that twinkle does not run anymore...

Frédéric Brière · Answer 8 · Sun Oct 15 2023 09:04:05 GMT+0800 (China Standard Time)

Finally a backtrace:

#6  0x00007fd405b51f89 in ?? () from /usr/lib64/alsa-lib/libasound_module_pcm_pipewire.so

Drat! I did not expect PipeWire to show up this early in the stack...

Could you please generate another backtrace after installing the debuginfo packages for libpipewire, pipewire-alsa and pipewire-spa-plugins? Thanks in advance!

Frédéric Brière · Answer 9 · Sun Oct 15 2023 09:15:13 GMT+0800 (China Standard Time)

@urbanware-org Like in my reply to @cyberbeat above, I also suggest you install the debuginfo packages for pipewire-libs and pipewire-alsa, just to be safe.

cyberbeat · Answer 10 · Mon Oct 16 2023 03:23:52 GMT+0800 (China Standard Time)

I also recognized this, and wait for the next crash (if there is any, after my pipewire update to pipewire 0.3.80)

cyberbeat · Answer 11 · Mon Oct 16 2023 16:43:21 GMT+0800 (China Standard Time)

Here is some more debuginfo:

Thread 1 (Thread 0x7f7d8f7fe700 (LWP 6267)):
#0  0x00007f7dca306c6b in raise () from /lib64/libc.so.6
#1  0x00007f7dca308305 in abort () from /lib64/libc.so.6
#2  0x00007f7dca2fec6a in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007f7dca2fecf2 in __assert_fail () from /lib64/libc.so.6
#4  0x00007f7dcd15d352 in snd_pcm_areas_copy (dst_areas=dst_areas@entry=0x7f7d8f7fd8e0, dst_offset=dst_offset@entry=0, src_areas=src_areas@entry=0x0, src_offset=src_offset@entry=1024, channels=channels@entry=1, frames=frames@entry=341, format=SND_PCM_FORMAT_S16_LE) at pcm.c:3341
#5  0x00007f7dcd15d3ff in snd_pcm_areas_copy_wrap (dst_channels=dst_channels@entry=0x7f7d8f7fd8e0, dst_offset=dst_offset@entry=0, dst_size=dst_size@entry=341, src_channels=0x0, src_offset=1024, src_size=4096, channels=1, frames=341, format=SND_PCM_FORMAT_S16_LE) at pcm.c:3432
#6  0x00007f7da5f01049 in snd_pcm_pipewire_process (pw=pw@entry=0x1f74e50, hw_avail=hw_avail@entry=0x7f7d8f7fd968, want=want@entry=341, b=<optimized out>, b=<optimized out>) at ../pipewire-alsa/alsa-plugins/pcm_pipewire.c:315
#7  0x00007f7da5f012a4 in on_stream_process (data=0x1f74e50) at ../pipewire-alsa/alsa-plugins/pcm_pipewire.c:440
#8  0x00007f7da473030d in impl_node_process_output (object=0x2ac3190) at ../src/pipewire/stream.c:1135
#9  0x00007f7da42a86e3 in ?? () from /usr/lib64/spa-0.2/audioconvert/libspa-audioconvert.so
#10 0x00007f7da4709239 in process_node (data=0x2ae5bb0) at ../src/pipewire/impl-node.c:1233
#11 node_on_fd_events (source=<optimized out>) at ../src/pipewire/impl-node.c:1306
#12 0x00007f7da47b709e in ?? () from /usr/lib64/spa-0.2/support/libspa-support.so
#13 0x00007f7da46dfbf0 in do_loop (user_data=0x1f70760) at ../src/pipewire/data-loop.c:65
#14 0x00007f7dcee3d6ea in start_thread () from /lib64/libpthread.so.0
#15 0x00007f7dca3d394f in clone () from /lib64/libc.so.6

Ralf Kilian · Answer 12 · Tue Oct 17 2023 02:01:12 GMT+0800 (China Standard Time)

I also suggest you install the debuginfo packages for pipewire-libs and pipewire-alsa, just to be safe.

@fbriere All right, I have installed the additional packages this morning.

Unfortunately, I can't provide a backtrace so far as Twinkle did not crash, yet.

cyberbeat · Answer 13 · Wed Oct 18 2023 23:02:06 GMT+0800 (China Standard Time)

I hope the next time, I can also deliver debuginfo for pipewire-spa calls.

Ralf Kilian · Answer 14 · Thu Oct 19 2023 23:08:04 GMT+0800 (China Standard Time)

So, Twinkle crashed once today and I can finally provide the backtrace.

Thread 45 "twinkle" received signal SIGABRT, Aborted.
[Switching to Thread 0x7fffaffff6c0 (LWP 431014)]
0x00007ffff52afecc in __pthread_kill_implementation () from /lib64/libc.so.6
#0  0x00007ffff52afecc in __pthread_kill_implementation () from /lib64/libc.so.6
#1  0x00007ffff525fab6 in raise () from /lib64/libc.so.6
#2  0x00007ffff52497fc in abort () from /lib64/libc.so.6
#3  0x00007ffff524971b in __assert_fail_base.cold () from /lib64/libc.so.6
#4  0x00007ffff5258696 in __assert_fail () from /lib64/libc.so.6
#5  0x00007ffff7accaac in snd_pcm_areas_copy () from /lib64/libasound.so.2
#6  0x00007ffff7accb53 in snd_pcm_areas_copy_wrap () from /lib64/libasound.so.2
#7  0x00007fffdc158006 in snd_pcm_pipewire_process.isra () from /lib64/alsa-lib/libasound_module_pcm_pipewire.so
#8  0x00007fffdc158132 in on_stream_process () from /lib64/alsa-lib/libasound_module_pcm_pipewire.so
#9  0x00007fffdc0c7710 in impl_node_process_input () from /lib64/libpipewire-0.3.so.0
#10 0x00007fffbc49bc34 in impl_node_process () from /usr/lib64/spa-0.2/audioconvert/libspa-audioconvert.so
#11 0x00007fffdc0ab1c9 in node_on_fd_events () from /lib64/libpipewire-0.3.so.0
#12 0x00007fffdc0268b6 in loop_iterate () from /usr/lib64/spa-0.2/support/libspa-support.so
#13 0x00007fffdc08a2a2 in do_loop () from /lib64/libpipewire-0.3.so.0
#14 0x00007ffff52ae19d in start_thread () from /lib64/libc.so.6
#15 0x00007ffff532fc60 in clone3 () from /lib64/libc.so.6

cyberbeat · Answer 15 · Fri Oct 20 2023 18:51:46 GMT+0800 (China Standard Time)

Now with all debug-info-packages (also pipewire libspa-support), using pipewire 0.3.82, libasound 1.2.6.1

Thread 1 (Thread 0x7f5d167fc700 (LWP 15928)):
#0  0x00007f5d5033ec6b in raise () from /lib64/libc.so.6
#1  0x00007f5d50340305 in abort () from /lib64/libc.so.6
#2  0x00007f5d50336c6a in __assert_fail_base () from /lib64/libc.so.6
#3  0x00007f5d50336cf2 in __assert_fail () from /lib64/libc.so.6
#4  0x00007f5d5319532e in snd_pcm_areas_copy (dst_areas=dst_areas@entry=0x0, dst_offset=dst_offset@entry=2276, src_areas=src_areas@entry=0x7f5d167fb900, src_offset=src_offset@entry=0, channels=channels@entry=1, frames=frames@entry=85, format=SND_PCM_FORMAT_S16_LE) at pcm.c:3340
#5  0x00007f5d531953ff in snd_pcm_areas_copy_wrap (dst_channels=0x0, dst_offset=2276, dst_size=4096, src_channels=src_channels@entry=0x7f5d167fb900, src_offset=src_offset@entry=0, src_size=src_size@entry=85, channels=1, frames=85, format=SND_PCM_FORMAT_S16_LE) at pcm.c:3432
#6  0x00007f5d2c3390ab in snd_pcm_pipewire_process (pw=pw@entry=0x7f5d04150d70, hw_avail=hw_avail@entry=0x7f5d167fb988, want=85, want@entry=3932, b=<optimized out>, b=<optimized out>) at ../pipewire-alsa/alsa-plugins/pcm_pipewire.c:321
#7  0x00007f5d2c3392a4 in on_stream_process (data=0x7f5d04150d70) at ../pipewire-alsa/alsa-plugins/pcm_pipewire.c:440
#8  0x00007f5d2469dcb8 in impl_node_process_input (object=0x7f5d040ac7d0) at ../src/pipewire/stream.c:1050
#9  0x00007f5d1c78b708 in impl_node_process (object=0x7f5d041682e8) at ../spa/plugins/audioconvert/audioadapter.c:1603
#10 0x00007f5d24674619 in process_node (data=0x7f5d04107280) at ../src/pipewire/impl-node.c:1225
#11 node_on_fd_events (source=<optimized out>) at ../src/pipewire/impl-node.c:1298
#12 0x00007f5d2c0eff7e in loop_iterate (object=object@entry=0x7f5d040b8098, timeout=timeout@entry=-1) at ../spa/plugins/support/loop.c:496
#13 0x00007f5d2464aed0 in do_loop (user_data=0x7f5d0412c2b0) at ../src/pipewire/data-loop.c:65
#14 0x00007f5d54e756ea in start_thread () from /lib64/libpthread.so.0
#15 0x00007f5d5040b94f in clone () from /lib64/libc.so.6

@fbriere Could you estimate, if this is an alsa bug, or pipewire bug?

Frédéric Brière · Answer 16 · Sun Oct 22 2023 09:59:56 GMT+0800 (China Standard Time)

Now with all debug-info-packages (also pipewire libspa-support), using pipewire 0.3.82, libasound 1.2.6.1

Thanks! I have opened an issue on the PipeWire tracker.

Konstantin K · Answer 17 · Mon Nov 06 2023 12:06:49 GMT+0800 (China Standard Time)

Should be fixed in the released 0.3.84 Pipewire

cyberbeat · Answer 18 · Tue Nov 07 2023 03:45:20 GMT+0800 (China Standard Time)

Yes, I already use 0.3.84, and did not have any crash until now. Many thanks @fbriere !

Ralf Kilian · Answer 19 · Thu Nov 09 2023 06:52:36 GMT+0800 (China Standard Time)

I assume that the problem might also be solved for me as well then. However, I cannot verify this (yet) as I no longer receive updates for Fedora 37 after the release of version 39 and don't have the latter running so far.

From this point of view, the issue could be closed, I guess, unless you explicitly want to wait for my confirmation.

Frédéric Brière · Answer 20 · Sat Nov 11 2023 11:11:37 GMT+0800 (China Standard Time)

So, Twinkle crashed once today and I can finally provide the backtrace.

Weird, your backtrace contains function names, but lacks argument values and line numbers. First time I've seen something like this. (scratches head) Still, it matches the one from @cyberbeat, and therefore was caused by the same PipeWire bug (now fixed).

Now, may I assume this was with an outgoing call? (It's okay if you don't remember after all this time. 😄) From your initial description, it seems that incoming calls would cause Twinkle to crash when attempting to play the ringtone, which does not involve PipeWire at all according to your configuration.

Would it be possible for you to post a backtrace of Twinkle crashing on an incoming call? (The lack of argument values and line numbers shouldn't pose too much of a problem here, since I have a good idea of where the bug is actually located, and am merely looking for confirmation.)

Frédéric Brière · Answer 21 · Sat Nov 11 2023 11:12:49 GMT+0800 (China Standard Time)

Yes, I already use 0.3.84, and did not have any crash until now. Many thanks @fbriere !

Glad to hear it, and you're welcome!

cyberbeat · Answer 22 · Sat Nov 11 2023 16:42:34 GMT+0800 (China Standard Time)

@fbriere could you have a look also at #323, I know this is hard to debug, but perhaps you have a suggestion, how I can investigate? I also could put some debug outputs in the code and compile an own version, if you give me some hints, where I should look for debugging?

Ralf Kilian · Answer 23 · Sat Nov 11 2023 21:10:00 GMT+0800 (China Standard Time)

Now, may I assume this was with an outgoing call? (It's okay if you don't remember after all this time. 😄)

Honestly, I do not remember. 😄

Would it be possible for you to post a backtrace of Twinkle crashing on an incoming call?

Sure, I can see what I can get. Within the last days it crashed a couple of times, so I am confident it will do again.

Ralf Kilian · Answer 24 · Wed Nov 15 2023 05:58:24 GMT+0800 (China Standard Time)

I assume that the problem might also be solved for me as well then. However, I cannot verify this (yet) as I no longer receive updates for Fedora 37 after the release of version 39 and don't have the latter running so far.

Interestingly, I meanwhile can get the PipeWire update to version 0.3.84 for Fedora 37, even though, its EOL is imminent.

Within the last days it crashed a couple of times, so I am confident it will do again.

Anyway, Twinkle did not crash again so far. The in-house upgrade to Fedora 39 at work is scheduled for in two days now. Maybe I can get some backtraces before that.

Ralf Kilian · Answer 25 · Thu Nov 23 2023 05:15:27 GMT+0800 (China Standard Time)

The in-house upgrade to Fedora 39 at work is scheduled for in two days now

The upgrade was performed last week and the systems are running on Fedora 39 with PipeWire version 0.3.85 now. Everything works as expected so far.

Before, Twinkle also did not crash anymore (still with PipeWire 0.3.83), so I unfortunately can't provide any further backtrace.

Ralf Kilian · Answer 26 · Fri Dec 01 2023 01:14:56 GMT+0800 (China Standard Time)

Everything works as expected so far.

Still the case, not a single crash.

For my part, this issue could be closed.