ffmepg version 5 together with libvmaf v2.3.1 shows discrepency on 10 bits videos

[Yixu Chen]

Testing on the same resolution Y4M video with pix_fmt yuv420p10le, the ffmpeg libvmaf output is different than the libvmaf CLI output.

With FFMPEG libvmaf

ffmpeg -y \
-i 4k_3M_Chasing1.y4m \
-i 4k_ref_Chasing1.y4m \
-lavfi "[0:v]setpts=PTS-STARTPTS[distorted]; \
[1:v]setpts=PTS-STARTPTS[reference]; \
[distorted][reference]libvmaf=log_fmt=json:log_path=ffmpeg_y4m.json:n_threads=8" \
-f null -

This give output

"vmaf": {
      "min": 40.457722,
      "max": 92.307341,
      "mean": 84.392485,
      "harmonic_mean": 83.634713
    }

With libvmaf CLI

vmaf \
--reference 4k_ref_Chasing1.y4m \
--distorted 4k_3M_Chasing1.y4m \
--json --threads 8 \
-o vmafcli.json

This gives output

"vmaf": {
      "min": 13.387595,
      "max": 87.385037,
      "mean": 75.670872,
      "harmonic_mean": 72.381078
    }

My software is ffmpeg version n5.1.1, libvmaf f266167 (should be v2.3.1).

If test videos are of yuv420p10le format, VMAF CLI, vmafossexec, VMAF python library give correct result, but ffmpeg libvmaf doesn’t.

If test videos are of yuv422p/yuv420p format, VMAF CLI and ffmpeg give the same result.

On Ubuntu and Mac, re-compiling vmaf and ffmpeg doesn’t correct this issue. On FFMPEG version 4, this issue doesn't exist.

Also, it seems unittest didn't pass when compiling the latest version v2.3.1.

[Kyle Swanson]

Can you confirm that both vmaf and ffmpeg are linking the exact same libvmaf? If that's the case and you are seeing numeric differences, then ffmpeg may be inserting some auto format conversion filters.

[Yixu Chen]

Yes. I installed libvmaf first and then compiled ffmpeg. I also replicated this issue on a separate EC2. Could you try replicated this on your end?
It seems FFMPEG have changed their API for version 5. Do you think this issue should be opened on ffmpeg instead of vmaf?

[Paul B Mahol]

you need 5.2 version, 5.1 does not have fix for 10bit libvmaf filter: https://git.videolan.org/?p=ffmpeg.git;a=commit;h=8357d4790fd0f11d7b76e20d321ec8241d80bd05

[Kyle Swanson]

you need 5.2 version, 5.1 does not have fix for 10bit libvmaf filter: https://git.videolan.org/?p=ffmpeg.git;a=commit;h=8357d4790fd0f11d7b76e20d321ec8241d80bd05

That's it. Thank you, Paul.

[Yixu Chen]

5.2 is not released yet. Thanks for confirming. Close this issue