Is it possible to generate a solution that only builds a few specific decoders and nothing else?

Question

Is it possible to generate a solution that only builds a few specific decoders and nothing else?

JPGygax68 opened this issue 2 years ago · comments

I have tried --disable-encoders and --disable-encoders followed by -enable-ffnvcodec but when I test with ffmpeg -codecs the codecs are all there, except with neither encoding nor decoding.

Is there a way to really remove the unneeded code?

Matthew Oliver · Answer 1 · Thu Mar 03 2022 14:44:06 GMT+0800 (China Standard Time)

I just checked and the above 3 commands do in fact disable all the encoders and decoders. There will still be some hwaccels as they are disabled with a different command --disable-hwaccels. You will also still have muxers/demuxers/parsers etc. so if you just want a specific decoders then you want to start with --disable-everything and then enable the desired decoder.

Hans-Peter Gygax · Answer 2 · Fri Mar 04 2022 18:08:30 GMT+0800 (China Standard Time)

Thank you @Sibras, that seems to work. Here is the relevant excerpt of the codec list:

 D.V.LS h264                 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_cuvid )
 D.V.L. hevc                 H.265 / HEVC (High Efficiency Video Coding) (decoders: hevc hevc_cuvid )

obtained with the configuration options:

SET PGOPTIONS=--disable-everything --disable-hwaccels --enable-ffnvcodec --enable-nvdec --enable-h264-cuvid-decoder --enable-h264-nvdec-hwaccel --enable-hevc-cuvid-decoder --enable-hevc-nvdec-hwaccel

A few questions, if I may:

The list still contains all known codecs, even if only two formats effectively have decoders. Where is that list stored, and can it be trimmed ? (only relevant if it lives in one of the libraries)
I effectively only need the nvcodecs, but the config process has apparently also enabled the software decoders; not that it matters much, but could that be avoided?

Matthew Oliver · Answer 3 · Sat Mar 05 2022 12:41:33 GMT+0800 (China Standard Time)

The list is hardcoded in, see avcodec/codec_desc.c. So it cant be removed or trimmed. One of the reasons for that is for codec identification so that it can identify which files it can support and which files it cant (i.e. error reporting on invalid files). Unfortunately this makes it non-trivial to remove.
Sometimes parts of the software codecs are still actually used even when using hwaccels as they have shared code between hardware/software that is used for things like passing packets/frames, or for more complex parts of decoding for situations where the hardware doesnt support the complete process. I havnt checked the exact command line your using but I believe thats the reason the software decoder is showing up.

Hans-Peter Gygax · Answer 4 · Sun Mar 20 2022 19:05:55 GMT+0800 (China Standard Time)

Thank you, I've now been able to configure my build to my wishes. One problem remains, that is how to build shared DLLs ? I've tried --enable-shared, but I got a message that this option was reserved and would be ignored.

Matthew Oliver · Answer 5 · Sun Mar 20 2022 22:07:26 GMT+0800 (China Standard Time)

In the created Visual Studio project just select a configuration with 'DLL' at the end as that will build a shared DLL. i.e. 'ReleaseDLL' for shared release builds while 'Release' is for static release builds.

Hans-Peter Gygax · Answer 6 · Mon Mar 21 2022 03:21:04 GMT+0800 (China Standard Time)

Thanks again @Sibras. Unfortunately, I'm getting linker errors in the ReleaseDLL configuration that I'm not getting with Release, such as _ff_inter_vlc. Any ideas?

Matthew Oliver · Answer 7 · Mon Mar 21 2022 06:10:11 GMT+0800 (China Standard Time)

Thanks again @Sibras. Unfortunately, I'm getting linker errors in the ReleaseDLL configuration that I'm not getting with Release, such as _ff_inter_vlc. Any ideas?

Hmmm, that would potentially be a bug, paste you exact command line here and ill see if I can reproduce

Hans-Peter Gygax · Answer 8 · Mon Mar 21 2022 06:59:57 GMT+0800 (China Standard Time)

This is the only part of the configuration script that I modified:

SET DEPENDENCIES=()
SET PGOPTIONS=--disable-everything --disable-hwaccels --enable-ffnvcodec --enable-nvdec --enable-h264-cuvid-decoder --enable-h264-nvdec-hwaccel --enable-hevc-cuvid-decoder --enable-hevc-nvdec-hwaccel --enable-avi-demuxer --enable-matroska-demuxer --enable-h264-demuxer --enable-hevc-demuxer --enable-mjpeg-demuxer --enable-mov-demuxer --enable-file-protocol --enable-tcp-protocol --enable-ffplay --enable-ffmpeg

Matthew Oliver · Answer 9 · Tue Mar 22 2022 06:17:27 GMT+0800 (China Standard Time)

This is the only part of the configuration script that I modified:

SET DEPENDENCIES=()
SET PGOPTIONS=--disable-everything --disable-hwaccels --enable-ffnvcodec --enable-nvdec --enable-h264-cuvid-decoder --enable-h264-nvdec-hwaccel --enable-hevc-cuvid-decoder --enable-hevc-nvdec-hwaccel --enable-avi-demuxer --enable-matroska-demuxer --enable-h264-demuxer --enable-hevc-demuxer --enable-mjpeg-demuxer --enable-mov-demuxer --enable-file-protocol --enable-tcp-protocol --enable-ffplay --enable-ffmpeg

This might actually be a bug on the upstream ffmpeg side due to config handling. Ill investigate it more when I get time. One thing ill point out is that --enable--ffplay requires sdl2 dependency but youve disabled that with --disable-eveything which might be confusing the config

Hans-Peter Gygax · Answer 10 · Tue Mar 29 2022 16:40:44 GMT+0800 (China Standard Time)

I've had a closer look at the problem, and it seems there is an odd effect happening when compiling msmpeg4data.c. The header file of that source file (msmpeg4data.h) contains several table (array) definitions - not just declarations as you would expect in a header file. And for some reason, those of those tables that are marked const end up as linker errors, meaning that the compiler does not generate those objects.

Not all those tables are marked const though (e.g. ff_mpeg4_rl_intra), and those that aren't are generated correctly. i.e. they don't cause linker errors.

Matthew Oliver · Answer 11 · Tue Mar 29 2022 23:15:22 GMT+0800 (China Standard Time)

OK seems it was an issue with config dependency handling which due to missing files in the upstream ffmpeg makefile meant that the file with those missing definitions wasnt being included in the build. A new version of FFVS has been pushed (1.10.2) that should be able to work around these issues.

Hans-Peter Gygax · Answer 12 · Wed Mar 30 2022 15:02:15 GMT+0800 (China Standard Time)

Thank you @Sibras , the DLLs now mostly build, except for libavformat and possibly libavdevice which depends on the former. The missing symbols are: _avpriv_dv_init_demux, _avpriv_dv_produce_packet, _avpriv_dv_get_packet and _ff_tls_open_underlying.

Matthew Oliver · Answer 13 · Thu Mar 31 2022 17:28:56 GMT+0800 (China Standard Time)

Hmm, ive pushed a new FFVS version to help here (1.10.3). But youll need some extra command line arguments to get it to work.
So youll need to add --disable-autodetect near the start and then --enable-cuvid and that works

Hans-Peter Gygax · Answer 14 · Thu Mar 31 2022 21:56:51 GMT+0800 (China Standard Time)

My config script:

SET DEPENDENCIES=( ^
x264, ^
x265 ^
)
SET PGOPTIONS=--disable-autodetect --disable-everything --disable-hwaccels --enable-ffnvcodec --enable-cuvid --enable-nvdec --enable-h264-cuvid-decoder --enable-h264-nvdec-hwaccel --enable-hevc-cuvid-decoder --enable-hevc-nvdec-hwaccel --enable-mjpeg-decoder --enable-avi-demuxer --enable-matroska-demuxer --enable-h264-demuxer --enable-hevc-demuxer --enable-mjpeg-demuxer --enable-mov-demuxer --enable-file-protocol --enable-tcp-protocol --enable-ffplay --enable-ffmpeg

now gives me the following output:

Located FFmpeg source directory at "../../ffmpeg/"

Creating project_get_dependencies.bat...

Do you want to download/update the required dependency projects (Y/N)?n
Running project generator...
Project generator (this may take several minutes, please wait)...
  Passing configure file...
  Outputting config.h...
  Outputting avconfig.h...
  Outputting ffversion.h...
  Outputting enabled components file libavfilter/filter_list.c...
  Outputting enabled components file libavcodec/codec_list.c...
  Outputting enabled components file libavcodec/parser_list.c...
  Outputting enabled components file libavcodec/bsf_list.c...
  Outputting enabled components file libavformat/demuxer_list.c...
  Outputting enabled components file libavformat/muxer_list.c...
  Outputting enabled components file libavdevice/indev_list.c...
  Outputting enabled components file libavdevice/outdev_list.c...
  Outputting enabled components file libavformat/protocol_list.c...
  Generating from Makefile (../../ffmpeg/libavdevice/MakeFile)...
  Generating missing DCE symbols (libavdevice)...
  Generating project exports file (libavdevice)...
  Generating project file (libavdevice)...
  Generating from Makefile (../../ffmpeg/libavfilter/MakeFile)...
  Generating from Makefile (../../ffmpeg//libavfilter/dnn/Makefile)...
  Generating from Makefile (../../ffmpeg/libavfilter/x86/MakeFile)...
  Generating missing DCE symbols (libavfilter)...
  Generating project exports file (libavfilter)...
  Generating project file (libavfilter)...
  Generating from Makefile (../../ffmpeg/libswscale/MakeFile)...
  Generating from Makefile (../../ffmpeg/libswscale/x86/MakeFile)...
  Generating missing DCE symbols (libswscale)...
  Generating project exports file (libswscale)...
  Generating project file (libswscale)...
  Generating from Makefile (../../ffmpeg/libpostproc/MakeFile)...
  Generating missing DCE symbols (libpostproc)...
  Generating project exports file (libpostproc)...
  Generating project file (libpostproc)...
  Generating from Makefile (../../ffmpeg/libavformat/MakeFile)...
  Generating missing DCE symbols (libavformat)...
  Generating project exports file (libavformat)...
Invalid switch - "".
  Error: Errors detected during compilation :-
         error C1083: Cannot open include file: 'bzlib.h': No such file or directory
         Based on the above error(s) there are files required for dependency libraries that are not available
         Ensure that any required dependencies are available in 'OutDir' based on the supplied configuration options before trying again.
         Consult the supplied readme for instructions for installing varying dependencies.
         If a dependency has been cloned from a ShiftMediaProject repository then ensure it has been successfully built before trying again.
           Removing the offending configuration option can also be used to remove the error.
Press any key to continue . . .

Matthew Oliver · Answer 15 · Fri Apr 01 2022 08:02:24 GMT+0800 (China Standard Time)

That was because one of the protocols was directly enabling bzip as a requested feature and you dont have that installed. I have created another! FFVS release that changes the order of config operations so that the disable-autodetect line activates sooner and enabling an option performs a deep enable (slightly different behavior to upstream ffmpeg but makes it simpler to enable individual components)

Hans-Peter Gygax · Answer 16 · Fri Apr 01 2022 15:40:29 GMT+0800 (China Standard Time)

Thank you once again @Sibras, I was able to make it work now. I had to add the switch --enable-dv-demux to get rid of three remaining undefined symbols, but now I'm ready to test my build.

Regarding bzip2, I had tried adding that dependency but it didn't seem to make a difference. No matter, it works now.

Matthew Oliver · Answer 17 · Sat Apr 02 2022 11:15:17 GMT+0800 (China Standard Time)

Thank you once again @Sibras, I was able to make it work now. I had to add the switch --enable-dv-demux to get rid of three remaining undefined symbols, but now I'm ready to test my build.

Hmm, it worked fine on mine without needing --enable-dv-demux, but I guess it depends on what version of ffmpeg source youre using. When disabling such large number of options then only enabling a few the upsteam ffmpeg source itself has many errors that can occur as there are so many possible options that they are not all tested. Many of these FFVS cant fix by itself as it depends on what in ffmpeg.

Regarding bzip2, I had tried adding that dependency but it didn't seem to make a difference. No matter, it works now

Yeah its not just a matter of adding it as a dependency but you also have to build it so the headers are installed before running FFVS