rkhamilton / vqgan-clip-generator

Implements VQGAN+CLIP for image and video generation, and style transfers, based on text and image prompts. Emphasis on ease-of-use, documentation, and smooth video creation.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Unknown encoder 'libx265' from FFmpeg

gateway opened this issue · comments

I'm not sure if this is a windows 10 thing but the conda install of your original ffmpeg seemed to now pick up the libx265 for some reason. I had to install this version to get it to work

conda install -c conda-forge ffmpeg

can we also choose h264 as well?

Hmm, well honestly sticking with h264 is probably to the best for all. Of course you could go crazy and do various codecs but its the biggest standard.

also I havent looked at your code but make sure your using gpu encoding as well..

Here is an example

ffmpeg -f image2 -framerate 30 -i [filename]_%04d.png -c:v h264_nvenc -preset slow -qp 18 -pix_fmt yuv420p [outputname].mp4

of course you can use -vf to scale as well...

Old article.. https://blog.henlo.co/2019/02/05/encoding-an-image-sequence-to-video/

In my experience nvenc videos have larger filesize and lower quality compared to libx264. NVenc is much closer to x265 if you are generating HEVC video, but the gap seems bigger in h.264. NVEnc also doesn't respect some of the video encoding options that you pass to ffmpeg (slow, fast, etc, crf). It's great for real time encoding, but I don't personally use it for archival stuff.

For example, I just used your command on a folder of generated stills
ffmpeg -f image2 -framerate 30 -i %d.png -c:v h264_nvenc -preset slow -qp 18 -pix_fmt yuv420p nvenc_h264_slow_qp18.mp4
ffmpeg -f image2 -framerate 30 -i %d.png -c:v libx264 -preset slow -qp 18 -pix_fmt yuv420p libx264_slow_qp18.mp4
The nvenc file is 17.4 MB, and the libx264 file is 2.9 MB.

As a suggestion, you may want to look into using CRF CRF instead of QP to control size and quality. That's the recommended control for quality/size.
https://trac.ffmpeg.org/wiki/Encode/H.264
ffmpeg -f image2 -framerate 30 -i %d.png -c:v libx264 -preset slow -crf 28 -pix_fmt yuv420p libxh264_slow_crf28.mp4
Of course, this is one where for some reason NVenc doesn't respect the parameter.
ffmpeg -f image2 -framerate 30 -i %d.png -c:v h264_nvenc -preset slow -crf 28 -pix_fmt yuv420p nvenc_h264_slow_crf28.mp4
ffmpeg -f image2 -framerate 30 -i %d.png -c:v h264_nvenc -preset slow -crf 16 -pix_fmt yuv420p nvenc_h264_slow_crf16.mp4
Both of these give me the same file size.

With the v1.1 update that I just published you can easily use nvenc if you like though, either by passing it as a vcodec argument
video_tools.encode_video(output_file=generated_video_no_audio,
path_to_stills=video_frames_to_encode,
metadata=text_prompts,
output_framerate=output_framerate,
assumed_input_framerate=extraction_framerate,
vcodec='h264_nvenc ')

Or just running your own ffmpeg command before the encoder command. Inserting the gap between generating frames and encoding video opens the flexibility to do whatever you want to generate video.
!ffmpeg -f image2 -framerate 30 -i [filename]_%04d.png -c:v h264_nvenc -preset slow -qp 18 -pix_fmt yuv420p [outputname].mp4

Let me know how it works out! I think this will give a lot of useful flexibility.

Instructions for libx265 issue were added to README.md. Closing.