libvmaf with CUDA -- how to build and use
DonTequila opened this issue · comments
Is there a documentation explaining how to build libvmaf with support for CUDA and how it's been used?
Hi all, can somebody please point me to some documentation about building libvmaf with CUDA support if it exists? Thanks!
Also, do I need a NVIDIA card for building it?
You don't need one for building, but you will need a card for executing. Also you need driver stub libraries, which are included in a CUDA docker container.
Could you ldd ffmpeg to see if it really loads the correct libraries ? Whenever that happened it just loaded the wrong .so for me.
Wait did you apply https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20230807/29cf249a/attachment.obj to ffmpeg ? See #1232.
I successfully built it on Linux with
meson setup -Denable_cuda=true libvmaf/build libvmafninja -C libvmaf/build install
sudo ln -s /opt/cuda/ /usr/local/cudawas also required on my operating system before those two commands, but that's distribution specific. I use Arch Linux.Standalone vmaf now successfully uses CUDA, by default.
However, vmaf in ffmpeg and aomenc don't... Both are freshly compiled.
I tried that however I still got a compilation error:
`[183/206] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o
FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o
nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Xcompiler=-fPIC -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o -c ../src/feature/cuda/integer_adm/adm_cm.cu
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
435 | function(_Functor&& __f)
| ^
/usr/include/c++/11/bits/std_function.h:435:145: note: ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
530 | operator=(_Functor&& __f)
| ^
/usr/include/c++/11/bits/std_function.h:530:146: note: ‘_ArgTypes’
[187/206] Compiling C object test/test_cambi.p/test_cambi.c.o
../test/test_cambi.c: In function ‘get_sample_image’:
../test/test_cambi.c:59:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
59 | int err, count = 0;
| ^~~
../test/test_cambi.c: In function ‘get_sample_image_8b’:
../test/test_cambi.c:82:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
82 | int err, count = 0;
| ^~~
../test/test_cambi.c: In function ‘get_sample_image_8x8’:
../test/test_cambi.c:94:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
94 | int err, count = 0;
| ^~~
[189/206] Compiling C++ object test/test_predict.p/.._src_svm.cpp.o
ninja: build stopped: subcommand failed.`
Any guess ?
I've been trying to build libvmaf with CUDA support into a Windows build that's being cross-compiled on an Ubuntu WSL / Docker container. I was doing this through ffmpeg-windows-build-helpers.
After installing CUDA (I tried 10.2, 11.6, 11.8, and 12.2) and NVCC, along with all the requirements for both libvmaf and the ffmpeg builder tool, I modified the ffmpeg build script to do the following three things:
- Added the
-Denable_cuda=trueflag to the libvmaf compile options - Added
--enable-nonfree --enable-ffnvcodecto the ffmpeg configuration options - Changed the script to pull the latest commit of the main branch rather than a specific release
After doing that, I get this issue:
The Meson build system
Version: 1.2.3
Source dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf
Build dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build
Build type: cross build
Project name: libvmaf
Project version: 2.3.1
C compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc (gcc 10.2.0 "x86_64-w64-mingw32-gcc (GCC) 10.2.0")
C linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc ld.bfd 2.36.1
C++ compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ (gcc 10.2.0 "x86_64-w64-mingw32-g++ (GCC) 10.2.0")
C++ linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ ld.bfd 2.36.1
C compiler for the build machine: cc (gcc 7.5.0 "cc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the build machine: cc ld.bfd 2.30
C++ compiler for the build machine: c++ (gcc 7.5.0 "c++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C++ linker for the build machine: c++ ld.bfd 2.30
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Check usable header "stdatomic.h" : YES
Configuring version.h using configuration
Program nasm found: YES (/usr/bin/nasm)
Configuring config.asm using configuration
Program xxd found: YES (/usr/bin/xxd)
Run-time dependency CUDA found: NO
src/meson.build:162:22: ERROR: Dependency lookup for cuda with method 'system' failed: Couldn't find requested CUDA module 'cudart_static'
A full log can be found at /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build/meson-logs/meson-log.txt
I fixed the above issue by copying the libcudart_static.a file that gets added to my system when installing Cuda 11.6 through the cuda-toolkit-11-6 package. For me this file is in both /usr/local/cuda-11.6/targets/x86_64-linux/lib and /usr/lib/x86_64-linux-gnu, and I copied this file to the ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/x86_64-w64-mingw32/lib directory.
After doing this, the cross_compile_ffmpeg.sh script to change what normally gets written to meson-cross.mingw.txt by changing the following section:
[binaries]
c = '${cross_prefix}gcc'
cpp = '${cross_prefix}g++'
ld = '${cross_prefix}ld'
ar = '${cross_prefix}ar'
strip = '${cross_prefix}strip'
pkgconfig = '${cross_prefix}pkg-config'
nm = '${cross_prefix}nm'
windres = '${cross_prefix}windres'
and adding cuda = '/usr/bin/nvcc' to the bottom.
Afterwards attempting to run the compile script again, I'm met with a different error, this one from libvmaf itself:
[2/19] Linking target test/test_picture.exe
FAILED: test/test_picture.exe
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ -o test/test_picture.exe test/test_picture.exe.p/test.c.obj test/test_picture.exe.p/test_picture.c.obj test/test_picture.exe.p/.._src_picture.c.obj test/test_picture.exe.p/.._src_mem.c.obj test/test_picture.exe.p/.._src_ref.c.obj test/test_picture.exe.p/.._src_thread_pool.c.obj -Wl,--allow-shlib-undefined -Wl,-O1 -pthread -Wl,--start-group -lcuda /home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libcudart_static.a -L/usr/local/cuda/lib/x64 -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lcuda
collect2: error: ld returned 1 exit status
[3/19] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
/usr/bin/nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj -c ../src/feature/cuda/integer_adm/adm_cm.cu
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign
/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
435 | function(_Functor&& __f)
| ^
/usr/include/c++/11/bits/std_function.h:435:145: note: ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
530 | operator=(_Functor&& __f)
| ^
/usr/include/c++/11/bits/std_function.h:530:146: note: ‘_ArgTypes’
ninja: build stopped: subcommand failed.
Not sure what the -lcuda option does specifically, but it seems to look for CUDA even though it was able to find it earlier.
I've relayed this info to both the ffmpeg-windows-build-helpers and the media-autobuild_suite repos. The former can't do much since they can't release builds with nonfree features. I'll check later today if the changes I listed above work with media-autobuild_suite
I successfully built it on Linux with
meson setup -Denable_cuda=true libvmaf/build libvmafninja -C libvmaf/build install
sudo ln -s /opt/cuda/ /usr/local/cudawas also required on my operating system before those two commands, but that's distribution specific. I use Arch Linux.
Standalone vmaf now successfully uses CUDA, by default.
However, vmaf in ffmpeg and aomenc don't... Both are freshly compiled.I tried that however I still got a compilation error:
`[183/206] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Xcompiler=-fPIC -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o -c ../src/feature/cuda/integer_adm/adm_cm.cu ../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign ../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’: 435 | function(_Functor&& __f) | ^ /usr/include/c++/11/bits/std_function.h:435:145: note: ‘_ArgTypes’ /usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’: 530 | operator=(_Functor&& __f) | ^ /usr/include/c++/11/bits/std_function.h:530:146: note: ‘_ArgTypes’ [187/206] Compiling C object test/test_cambi.p/test_cambi.c.o ../test/test_cambi.c: In function ‘get_sample_image’: ../test/test_cambi.c:59:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable] 59 | int err, count = 0; | ^~~ ../test/test_cambi.c: In function ‘get_sample_image_8b’: ../test/test_cambi.c:82:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable] 82 | int err, count = 0; | ^~~ ../test/test_cambi.c: In function ‘get_sample_image_8x8’: ../test/test_cambi.c:94:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable] 94 | int err, count = 0; | ^~~ [189/206] Compiling C++ object test/test_predict.p/.._src_svm.cpp.o ninja: build stopped: subcommand failed.`Any guess ?
I have the same problem, Ubuntu 22.04, CUDA 12.3. It seems this is a problem with gcc and nvcc compaibility (see here)
@damian101 I think i had your same issue. I compiled both vmaf and FFmpeg on Arch from the latest git version, however the filter libvmaf_cuda was missing from the freshly built ffmpeg.
It turns out that ninja -C libvmaf/build install installs to /usr/local/ (so /usr/local/include, /usr/local/lib and /usr/local/bin), ffmpeg on the other hand seems to look for libraries inside /usr/include and /usr/lib first.
It's thus very likely that you were compiling ffmpeg with the version of libvmaf that's included on the arch repos, which is compiled without -Denable_cuda=true.
Before compiling FFmpeg, take a look at the output of the configure script, you should see libvmaf_cuda listed under Enabled filters:, if that's not the case you either forgot to enable the necessary configurations or it's not using the right library.