How can I process video files directly,not picture sequence frame

Question

How can I process video files directly,not picture sequence frame

zhanghongyong123456 opened this issue 8 months ago · comments

xiaosi commented 8 months ago

Why can not input video directly for interpolation processing, only the image sequence file can be operated?

Jerry Hogsett · Answer 1 · Fri Nov 24 2023 21:14:41 GMT+0800 (China Standard Time)

@zhanghongyong123456

Why can not input video directly for interpolation processing, only the image sequence file can be operated?

Try the Video Remixer tab. It's designed for starting initially with a video (optionally + audio) and applying various processing, including Video Inflation.

The software uses PNG files internally to handle video losslessly. The tools in the other tabs are designed to operate on these sets of PNG files.

Also, visit the Tools tab, and in particular, the File Conversion tab. There are tools to convert between video and PNG formats.

xiaosi · Answer 2 · Sat Nov 25 2023 10:42:50 GMT+0800 (China Standard Time)

@zhanghongyong123456

Why can not input video directly for interpolation processing, only the image sequence file can be operated?

Try the Video Remixer tab. It's designed for starting initially with a video (optionally + audio) and applying various processing, including Video Inflation.

The software uses PNG files internally to handle video losslessly. The tools in the other tabs are designed to operate on these sets of PNG files.

Also, visit the Tools tab, and in particular, the File Conversion tab. There are tools to convert between video and PNG formats.

when i use interpolate video get image sequence,how to conbine use tool tab (PNG Sequence to MP4)

Jerry Hogsett · Answer 3 · Sat Nov 25 2023 14:33:22 GMT+0800 (China Standard Time)

@zhanghongyong123456 I'm sorry you got this error. It's easy to work around. It is because FFmpeg requires files to be named with sequential index numbers. However the Video Inflation feature intentionally does not automatically re-index the files, given the existing names can be used to determine which original frame the interpolated frames were made from.

To Fix: use the Tools -> Resequence Files feature. It will get the list of files, then re-index them properly. Then PNGtoMP4 can be used without further issue.

I recommend using the Video Remixer tab, which is designed to integrate all the low-level workings together in a smooth experience.

xiaosi · Answer 4 · Sat Nov 25 2023 14:46:09 GMT+0800 (China Standard Time)

@zhanghongyong123456 I'm sorry you got this error. It's easy to work around. It is because FFmpeg requires files to be named with sequential index numbers. However the Video Inflation feature intentionally does not automatically re-index the files, given the existing names can be used to determine which original frame the interpolated frames were made from.

To Fix: use the Tools -> Resequence Files feature. It will get the list of files, then re-index them properly. Then PNGtoMP4 can be used without further issue.

I recommend using the Video Remixer tab, which is designed to integrate all the low-level workings together in a smooth experience.

thanks,i try it ,
I found that the original project two scripts(demo_2x.py and demo_Nx.py) used different models(['ours', 'ours_small'] and ['ours_t', 'ours_small_t'] ),Whether 2x or Nx this project is implemented on only one model(ours)

Jerry Hogsett · Answer 5 · Sat Nov 25 2023 15:29:15 GMT+0800 (China Standard Time)

Thanks @zhanghongyong123456

If you look in config.yaml there is a section that allows you to configure which model is used by EMA-VFI as follows:

engine_settings:
  gpu_ids: "0"
  model: "ours"
  use_time_step: False

Jerry Hogsett · Answer 6 · Sat Nov 25 2023 15:31:17 GMT+0800 (China Standard Time)

Note: When

use_time_step: true

It automatically added the "_t" to the model name that it will select and use. I have found that the time-step version of the model doesn't suit my needs, but I allow it to be enabled in the software.

xiaosi · Answer 7 · Sat Nov 25 2023 15:33:53 GMT+0800 (China Standard Time)

Note: When
use_time_step: true
It automatically added the "_t" to the model name that it will select and use. I have found that the time-step version of the model doesn't suit my needs, but I allow it to be enabled in the software.

Thanks for the answer, what is the difference between several models

Jerry Hogsett · Answer 8 · Sat Nov 25 2023 15:40:11 GMT+0800 (China Standard Time)

Thanks for the answer, what is the difference between several models

My understanding is the main model "ours" is strictly for 50% interpolation between two frames, and the time-step version "ours_t" allows interpolation at any percentage between two frames. Also there are small versions of the models as well, which I don't use.

I found the time-step model to be insufficient compared to my binary-search (Frame Search feature) method of achieving interpolation at other than the 50% point.

I recommend reaching out to the EMA-VFI folks directly for a more precise technical answer.

xiaosi · Answer 9 · Mon Nov 27 2023 10:40:44 GMT+0800 (China Standard Time)

Thanks for the answer, what is the difference between several models

My understanding is the main model "ours" is strictly for 50% interpolation between two frames, and the time-step version "ours_t" allows interpolation at any percentage between two frames. Also there are small versions of the models as well, which I don't use.

I found the time-step model to be insufficient compared to my binary-search (Frame Search feature) method of achieving interpolation at other than the 50% point.

I recommend reaching out to the EMA-VFI folks directly for a more precise technical answer.

Thank you very much for your clarification,