In FFmpeg, when the -phone-model option is enabled, the VMAF does not change

[Olon11]

I am using FFmpeg 6 to compare the visual quality of different encoding parameters. To compute the VMAF, I use this FFmpeg command :

ffmpeg -hide_banner -i distorted.mp4 -i original.y4m -lavfi libvmaf=feature=name=psnr:phone_model=1:n_threads=8:log_path=vmaf.json:log_fmt=json -f null -

However, when I remove the phone_model=1 option, the VMAF does not change.

Is this normal?

[Nil Fons Miret]

Hi, sorry for the delayed response. I believe the problem is in the use of the option phone_model, instead of the new syntax specifying a model with enable_transform=true. Refer to the ffmpeg documentation for more details, but I was able to get different scores with the following commands:

# Regular VMAF (score: 83.452746 on my inputs)
ffmpeg -hide_banner -i distorted.mp4 -i original.y4m -lavfi libvmaf='feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1:log_path=vmaf.json:log_fmt=json' -f null -

# Phone VMAF (score: 96.661642 on my inputs)
ffmpeg -hide_banner -i distorted.mp4 -i original.y4m -lavfi libvmaf='feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1\\:enable_transform=true:log_path=vmaf.json:log_fmt=json' -f null -

Note the single quotes enclosing the libvmaf command as well. On the regular VMAF command, it is not necessary to specify the model as it will default to v0.6.1 anyway but I just wrote it to be explicit.

[Olon11]

Thank you for your answer. Adding the model=version=vmaf_v0.6.1\\:enable_transform=true options to the command gives me a different score.

[Ali Ak]

Hi, sorry for the delayed response. I believe the problem is in the use of the option phone_model, instead of the new syntax specifying a model with enable_transform=true. Refer to the ffmpeg documentation for more details, but I was able to get different scores with the following commands:

# Regular VMAF (score: 83.452746 on my inputs)
ffmpeg -hide_banner -i distorted.mp4 -i original.y4m -lavfi libvmaf='feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1:log_path=vmaf.json:log_fmt=json' -f null -

# Phone VMAF (score: 96.661642 on my inputs)
ffmpeg -hide_banner -i distorted.mp4 -i original.y4m -lavfi libvmaf='feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1\\:enable_transform=true:log_path=vmaf.json:log_fmt=json' -f null -

I am having issues running the vmaf phone model this way.
The first comment returns the VMAF score as expected.
Second comment returns the following error:
Error applying option 'enable_transform' to filter 'libvmaf': Option not found
I have tried both in ffmpeg 6 and 7 builds.

[Nil Fons Miret]

Hi @kyillene, this seems like the option enable_transform isn't being parsed as part of the model, but as an independent option to the libvmaf filter. This would point to the escape mechanism \\: not working on your system.

To help debug:

1. What is your OS?
2. Could you paste the output of this command?

ffmpeg -loglevel debug -i nonexistent -i nonexistent -lavfi libvmaf='feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1\\:enable_transform=true:log_path=vmaf.json:log_fmt=json' -f null -             

the inputs don't matter, I just set it to some file that doesn't exist so that the output is cut short. That should give us some information about how ffmpeg is parsing your command line. For example, the most relevant lines for me are:

[...]
Reading option '-lavfi' ... matched as option 'lavfi' (create a complex filtergraph) with argument 'libvmaf=feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1\\:enable_transform=true:log_path=vmaf.json:log_fmt=json'.
[AVFilterGraph @ 0x600002c64060] Setting 'feature' to value 'name=psnr'
[AVFilterGraph @ 0x600002c64060] Setting 'n_threads' to value '8'
[AVFilterGraph @ 0x600002c64060] Setting 'model' to value 'version=vmaf_v0.6.1:enable_transform=true'
[AVFilterGraph @ 0x600002c64060] Setting 'log_path' to value 'vmaf.json'
[AVFilterGraph @ 0x600002c64060] Setting 'log_fmt' to value 'json'
[...]

where enable_transform is parsed as part of model.

[Nil Fons Miret]

I would also suggest trying to replace the double backslashes (\\:) with a single backslash (\:) or even with a quadruple backslash (\\\\:).

[Ali Ak]

Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-i' ... matched as input url with argument 'nonexistent'.
Reading option '-i' ... matched as input url with argument 'nonexistent'.
Reading option '-lavfi' ... matched as option 'lavfi' (create a complex filtergraph) with argument 'libvmaf='feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1\\:enable_transform=true:log_path=vmaf.json:log_fmt=json''.
Reading option '-f' ... matched as option 'f' (force format) with argument 'null'.
Reading option '-' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Applying option lavfi (create a complex filtergraph) with argument libvmaf='feature=name=psnr:n_threads=8:model=version=vmaf_v0.6.1\\:enable_transform=true:log_path=vmaf.json:log_fmt=json'.
Successfully parsed a group of options.
Parsing a group of options: input url nonexistent.
Successfully parsed a group of options.
Opening an input file: nonexistent.

Thank you for the swift response.
The output is as above with double backslashes on windows 10 with the ffmpeg 6.

[Ali Ak]

Now I tried with a single backslash and it worked. The VMAF score mapping between the phone and 1080p predictions seems reasonable as well. Thanks for the quick response again.

[Nil Fons Miret]

Awesome, this command line escaping is a pain point we are aware of, as it can vary across OS and shells. I believe other ffmpeg filters have the same issue: see the examples here. We can probably do a better job at documenting this.