SHI-Labs / VideoINR-Continuous-Space-Time-Super-Resolution

[CVPR 2022] VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution

VideoINR

This repository contains the official implementation for VideoINR introduced in the following paper:

VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution
Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, Xiaolong Wang
CVPR 2022

You can find more visual results and a brief introduction to VideoINR at our project page.

Citation

If you find our work useful in your research, please cite:

@inproceedings{chen2022vinr,
  author    = {Chen, Zeyuan and Chen, Yinbo and Liu, Jingwen and Xu, Xingqian and Goel, Vidit and Wang, Zhangyang and Shi, Humphrey and Wang, Xiaolong},
  title     = {VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution},
  journal   = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year      = {2022},
}

Environmental Setup

The code is tested in:

• Python 3.6
• Pytorch 1.6.0
• Cuda 10.1
• Deformable Convolution v2. Following Zooming Slowmo, we adopt CharlesShang's implementation in the submodule.

If you are using Anaconda, the following command can be used to build the environment:

conda create -n videoinr
conda activate videoinr
conda install pytorch=1.6 -c pytorch

cd models/modules/DCNv2/
python setup.py install

Demo

1. Download the pre-trained model from google drive.

2. Convert your video of interest to a sequence of images. This process can be completed by many apps, e.g. ffmpeg and AdobePR.

The folder that contains this image sequence should have a structure as follows:

data_path
├── img_1.jpg
├── img_2.jpg
├── ...
├── img_n.jpg

3. Using VideoINR for performing space-time super-resolution. You can adjust up-sampling scales by setting different space_scale and time_scale.

python demo.py --space_scale 4 --time_scale 8 --data_path [YOUR_DATA_PATH]

4. The output would be three folders including low-resolution images, bicubic-upsampling images, and the results of VideoINR.

Training