When using hardware encoding, may crash or produce videos with pixel misalignment.
xiaohuirong opened this issue · comments
I use the options -c hevc_vaapi -g "<region>" to record windows, and different region selections result in different behaviors for wl-recorder. Running wf-recorder -c hevc_vaapi -g "683,48 1333x140" -f crash.mkv causes the program to crash. wf-recorder -c hevc_vaapi -g "642,317 1322x807" -f misalignment.mkv results in pixel misalignment. However, wf-recorder -c hevc_vaapi -g "637,317 1333x807" -f normal.mkv works correctly. I have tried all supported hardware encoders, and the issue persists. If using software encoding, it works correctly regardless of the selected region.
https://youtu.be/nmSDx5srsY8?si=ELj0hTUGszMeuwRm
- OS: Archlinux 6.4.12-zen1-1-zen
- WM: Wayfire lasted git version
- CPU: i5-13400 / i7-7500U
- GPU: Intel UHD Graphics 730 / Intel HD Graphics 620, both have this issue.
Does --no-dmabuf help?
Does
--no-dmabufhelp?
No effect, same problem.
My screen's resolution is 2560x1440.
A quick bisect shows the problem was introduced by #206, with a backtrace from ASAN:
==3972888==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f5fe09d3000 at pc 0x7f6001449d21 bp 0x7f5ff25fd6f0 sp 0x7f5ff25fcea0
WRITE of size 860160 at 0x7f5fe09d3000 thread T1
#0 0x7f6001449d20 in __interceptor_memcpy (/lib64/libasan.so.8+0x49d20)
#1 0x7f5fee919c8f in MosUtilities::MosSecureMemcpy(void*, unsigned long, void const*, unsigned long) (/usr/lib64/dri/iHD_drv_video.so+0x119c8f)
#2 0x7f5feeac355a in DdiMedia_PutImage(VADriverContext*, unsigned int, unsigned int, int, int, unsigned int, unsigned int, int, int, unsigned int, unsigned int) (/usr/lib64/dri/iHD_drv_video.so+0x2c355a)
#3 0x7f6001b49cfb in vaPutImage (/lib64/libva.so.2+0xdcfb)
#4 0x7f600125e941 in vaapi_unmap_frame (/lib64/libavutil.so.57+0x35941)
#5 0x7f6001257c22 in ff_hwframe_unmap (/lib64/libavutil.so.57+0x2ec22)
#6 0x7f600124c601 in buffer_replace (/lib64/libavutil.so.57+0x23601)
#7 0x7f6001257a8b in av_frame_unref (/lib64/libavutil.so.57+0x2ea8b)
#8 0x7f6001257be9 in av_frame_free (/lib64/libavutil.so.57+0x2ebe9)
#9 0x7f600126075b in vaapi_transfer_data_to.lto_priv.0 (/lib64/libavutil.so.57+0x3775b)
#10 0x7f600125827f in av_hwframe_transfer_data (/lib64/libavutil.so.57+0x2f27f)
#11 0x7f5fff7efa79 in hwupload_filter_frame.lto_priv.0 (/lib64/libavfilter.so.8+0x1efa79)
#12 0x7f5fff8e3ae8 in ff_filter_graph_run_once.isra.0 (/lib64/libavfilter.so.8+0x2e3ae8)
#13 0x7f5fff73e76b in get_frame_internal (/lib64/libavfilter.so.8+0x13e76b)
#14 0x416e90 in FrameWriter::push_frame(AVFrame*, long) ../src/frame-writer.cpp:688
#15 0x417a08 in FrameWriter::add_frame(unsigned char const*, long, bool) ../src/frame-writer.cpp:743
#16 0x439806 in write_loop ../src/main.cpp:541
#17 0x43bd84 in operator() ../src/main.cpp:1191
#18 0x4402f9 in __invoke_impl<void, main(int, char**)::<lambda()> > /usr/include/c++/12/bits/invoke.h:61
#19 0x440275 in __invoke<main(int, char**)::<lambda()> > /usr/include/c++/12/bits/invoke.h:96
#20 0x440143 in _M_invoke<0> /usr/include/c++/12/bits/std_thread.h:258
#21 0x44008e in operator() /usr/include/c++/12/bits/std_thread.h:265
#22 0x44004c in _M_run /usr/include/c++/12/bits/std_thread.h:210
#23 0x7f5fff2dbc02 in execute_native_thread_routine (/lib64/libstdc++.so.6+0xdbc02)
#24 0x7f5ffe8ae14c in start_thread (/lib64/libc.so.6+0x8b14c)
#25 0x7f5ffe92f9ff in clone3 (/lib64/libc.so.6+0x10c9ff)
0x7f5fe09d3000 is located 6144 bytes to the left of 699312-byte region [0x7f5fe09d4800,0x7f5fe0a7f3b0)
allocated by thread T1 here:
#0 0x7f60014ba097 in calloc (/lib64/libasan.so.8+0xba097)
#1 0x7f5feef36787 in KernelDll_AllocateStates (/usr/lib64/dri/iHD_drv_video.so+0x736787)
#2 0x7f5feef2adf3 in vp::VpPlatformInterface::InitVPFCKernels(tagKdll_RuleEntry const*, unsigned int const*, unsigned int, unsigned int const*, unsigned int, void (*)(tagKdll_State*)) (/usr/lib64/dri/iHD_drv_video.so+0x72adf3)
#3 0x7f5feeed5e5e in vp::VpPlatformInterfaceG12Tgllp::InitVpRenderHwCaps() (/usr/lib64/dri/iHD_drv_video.so+0x6d5e5e)
#4 0x7f5feeee8446 in vp::VpFeatureManagerNext::Init(void*) (/usr/lib64/dri/iHD_drv_video.so+0x6e8446)
#5 0x7f5feef2cbed in vp::VpPipeline::Init(void*) (/usr/lib64/dri/iHD_drv_video.so+0x72cbed)
#6 0x7f5feeed92b4 in VpPipelineAdapterLegacy::Init(VpSettings const*, _VP_MHWINTERFACE) (/usr/lib64/dri/iHD_drv_video.so+0x6d92b4)
#7 0x7f5feeec13df in VpPipelineG12Adapter::Allocate(VpSettings const*) (/usr/lib64/dri/iHD_drv_video.so+0x6c13df)
#8 0x7f5feef59e2a in DdiVp_CreateContext(VADriverContext*, unsigned int, int, int, int, unsigned int*, int, unsigned int*) [clone .constprop.0] (/usr/lib64/dri/iHD_drv_video.so+0x759e2a)
#9 0x7f5feeac3472 in DdiMedia_PutImage(VADriverContext*, unsigned int, unsigned int, int, int, unsigned int, unsigned int, int, int, unsigned int, unsigned int) (/usr/lib64/dri/iHD_drv_video.so+0x2c3472)
#10 0x7f6001b49cfb in vaPutImage (/lib64/libva.so.2+0xdcfb)
Thread T1 created by T0 here:
#0 0x7f600144b3e6 in __interceptor_pthread_create (/lib64/libasan.so.8+0x4b3e6)
#1 0x7f5fff2dbcd8 in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/lib64/libstdc++.so.6+0xdbcd8)
#2 0x43e8af in main ../src/main.cpp:1190
#3 0x7f5ffe84a50f in __libc_start_call_main (/lib64/libc.so.6+0x2750f)
SUMMARY: AddressSanitizer: heap-buffer-overflow (/lib64/libasan.so.8+0x49d20) in __interceptor_memcpy
Shadow bytes around the buggy address:
0x0fec7c1325b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0fec7c1325c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0fec7c1325d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0fec7c1325e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0fec7c1325f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0fec7c132600:[fa]fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0fec7c132610: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0fec7c132620: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0fec7c132630: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0fec7c132640: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0fec7c132650: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==3972888==ABORTING
This also appears to be related to the geometry, so there is probably a mismatch between the passed sizes somewhere.
cc @nowrep
With geometry the dmabuf path isn't taken, so I guess something broke in add_frame?
I can't reproduce the crash / garbled output with mesa, but I noticed one issue: the sw_format was set incorrectly when using hw encoding without dma-buf capture.
Please post full log wf-recorder -l ... with current master and at ffcb6f9
From #232 (comment) it looks like the hwfc width/height is somehow wrong?
I just checked your PR and the crash is gone (I can also reproduce the crash on my 12700, didn't have access to it for some time). @xiaohuirong I hope the crash is fixed for you too?
I'm very happy the issue has been fixed, thank you all.
The issue seems to persist when the scaling is set to 1.25 on my 2560x1440 screen.
wf-recorder -g "110,93 800x600" -c hevc_vaapi will result in a chaotic video.
Please try #242 and if that doesn't help also this patch on top of it:
diff --git a/src/frame-writer.cpp b/src/frame-writer.cpp
index 9e28313..197802a 100644
--- a/src/frame-writer.cpp
+++ b/src/frame-writer.cpp
@@ -262,6 +262,7 @@ void FrameWriter::init_video_filters(const AVCodec *codec)
exit(-1);
}
+#if 0
AVBufferSrcParameters *p = av_buffersrc_parameters_alloc();
memset(p, 0, sizeof(*p));
p->format = AV_PIX_FMT_NONE;
@@ -272,6 +273,7 @@ void FrameWriter::init_video_filters(const AVCodec *codec)
std::cerr << "Cannot set hwcontext filter in: " << averr(err) << std::endl;;
exit(-1);
}
+#endif
err = avfilter_graph_create_filter(&this->videoFilterSinkCtx, sink, "Sink",
NULL, NULL, this->videoFilterGraph);Can you also reproduce on 714f19c, to confirm regression?
I encountered the following error.
wf-recorder -g "110,93 800x600" -c hevc_vaapi
selected region 110,93 800x600
Using video filter: hwupload,scale_vaapi=format=nv12
[hwupload @ 0x7f0dbc005540] A hardware device reference is required to upload frames to.
[Parsed_hwupload_0 @ 0x7f0dbc005440] Query format failed for 'Parsed_hwupload_0': Invalid argument
Failed to configure graph filter: %
And what build is this? What commit / patch applied?
I added the parameter -d /dev/dri/renderD128, and it runs now, but there are still issues with the video.
Yeah the older versions need to explicitly set device.
Please open new issue as this is not related to dmabuf capture, thanks.
Yeah the older versions need to explicitly set device.
Please open new issue as this is not related to dmabuf capture, thanks.
Ok