MCG-NJU / EMA-VFI

[CVPR 2023] Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolatio

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Running out of ram on large 4k images

luuude opened this issue · comments

Hi and conratulations on this awesome work!

I have been experimenting with this abit and I run out of ram when using the large model on 4k images when running on 24GB gpu

Any ideas on how to process large images on 24GB?

I am going to test splitting it up i four pieces and reassembling but I fear there will be seams.

@luuude I have successfully processed 4K content on my RTX 3090 w/ 24GB, BUT:

  • each frame takes about 90+ seconds to process, in some cases, twice that
  • it only works if I enable the NVIDIA feature to overflow into system RAM
  • My system has 64 GB system RAM and it allows the GPU to use 32 GB
  • While interpolating, the total memory load on the GPU is 56GB!

I've concluded it's not worth interpolating 4K content with this hardware.

I've also thought about tiling, but that would likely interfere with the engine's ability to recognize objects and estimate motion, creating odd seams.

90 seconds is quite rough but for VFX rendering not that bad. Usually I work with quite short cuts under 100 frames.
How do you enable the NVIDIA feature to overflow into system RAM?

In the "NVIDIA Control Panel" under "Manage 3D Settings" I have this option:

"CUDA - Sysmem Fallback Policy"