Question: Video HW Decoder

Question

Question: Video HW Decoder

proconsule opened this issue 3 years ago · comments

Hi there is any chance to get Video HW decoder?
I know Tegra can decode h264/hevc in hw, any resarch on getting it in libnx?
Thanks

MRiCEQB · Answer 1 · Sun Nov 06 2022 02:41:00 GMT+0800 (China Standard Time)

Any update on this?

Palidino · Answer 2 · Mon Feb 06 2023 10:07:05 GMT+0800 (China Standard Time)

Any plans for this ever?

Bennett Blodinger · Answer 3 · Tue Apr 04 2023 00:36:33 GMT+0800 (China Standard Time)

I've started a bounty for this to be implemented.
https://app.bountysource.com/issues/116504826-question-video-hw-decoder

averne · Answer 4 · Sun Aug 20 2023 00:10:21 GMT+0800 (China Standard Time)

I've released a FFmpeg fork enabling acceleration for all codecs supported by the hardware. I don't feel this really belongs in libnx, as most of it is not specific to the Switch/Horizon OS.
https://github.com/averne/FFmpeg/tree/tx1

Bennett Blodinger · Answer 5 · Sun Aug 20 2023 00:12:25 GMT+0800 (China Standard Time)

I agree that this shouldn't be in libnx. @proconsule, can you close this so I can award @averne the bounty?

proconsule · Answer 6 · Sun Aug 20 2023 03:29:59 GMT+0800 (China Standard Time)

I have to try this out on NXMP replacing the ffmpeg in use.

Thanks for this great result

averne · Answer 7 · Sun Aug 20 2023 03:43:04 GMT+0800 (China Standard Time)

I have to try this out on NXMP replacing the ffmpeg in use.

Thanks for this great result

You can contact me on discord @avhe if you run into issues.

Basic integration isn't hard, but for optimal performance you would need a deko3d graphics backend for mpv. In OpenGL, you are unable to directly import frames from the decoder, so the following happens:
decoder output (block linear) -> copied to staging buffer (pitch linear) using VIC -> copied to GPU texture (block linear) using GPU copy engine.

With deko3d, you just need to create a texture with DkImageFlags_UsageVideo over the frame data.

Dave Murphy · Answer 8 · Sun Aug 20 2023 05:42:59 GMT+0800 (China Standard Time)

I've released a FFmpeg fork enabling acceleration for all codecs supported by the hardware. I don't feel this really belongs in libnx, as most of it is not specific to the Switch/Horizon OS. https://github.com/averne/FFmpeg/tree/tx1

Would be awesome if you got this upstreamed to FFmpeg but also PR'd needed changes to dkP packages repo. I've put a quick test @ https://github.com/devkitPro/pacman-packages/tree/ffmpeg-update but I'm not 100% what configure options we need.

fwiw, that package can be installed using [sudo] [dkp-]pacman -U https://pkg.devkitpro.org/packages/switch-ffmpeg-6.0-1-any.pkg.tar.zst if anyone wants to test.

averne · Answer 9 · Sun Aug 20 2023 18:11:42 GMT+0800 (China Standard Time)

Would be awesome if you got this upstreamed to FFmpeg

I contacted the FFmpeg maintainers, not sure it could be upstreamed in its current state as it contains nvidia headers (even though they are under a permissive license). There are also a number of hacks, especially surface alignment, that would need some cleaning up.

I'm not 100% what configure options we need.

At minimum, you need to add --enable-gpl since the hwaccel is gated by this. I used the following flags, but I see the devkitPro package has some others. I'll merge those flags and put together a proper PR.

--cc=aarch64-none-elf-gcc --cxx=aarch64-none-elf-g++ --ar=aarch64-none-elf-ar --nm=aarch64-none-elf-nm --extra-cflags='-isystem /opt/devkitpro/libnx/include' --enable-asm --enable-gpl --enable-tx1 --enable-static --disable-shared --enable-zlib --enable-bzlib --enable-libass --enable-libfreetype --enable-libfribidi --disable-doc --disable-programs --target-os=horizon --enable-cross-compile --cross-prefix=aarch64-none-elf- --arch=aarch64 --cpu=cortex-a57 --enable-neon --enable-pic --disable-autodetect --disable-runtime-cpudetect --disable-debug