esw0116 / DynaVSR

DynaVSR: Dynamic Adaptive Blind VideoSuper-Resolution

DynaVSR

DynaVSR: Dynamic Adaptive Blind VideoSuper-Resolution

Suyoung Lee*, Myungsub Choi*, Kyoung Mu Lee

Seoul National University

This repository is the official implementation of DynaVSR, which is accepted in WACV2021.

Link: Paper(Arxiv) | Paper(CVFOpenAccess)

Directory Structure

project
│   README.md
└───dataset - make symbolic link of your data here
└───pretrained_models - put the downloadedpretrained model here
└───codes
│   └───data
│       │   common.py
│       │   data_sampler.py
│       │   old_kernel_generator.py
│       │   random_kernel_generator.py
│       └───baseline - for finetuning the VSR network
│       └───estimator - for training MFDN, SFDN
│       └───meta_learner - training maml network
│   └───data_scripts - miscellaneous scripts (same as EDVR)
│   └───metrics - metric calculation (same as EDVR)
│   └───models - model collections
│   └───options
│       └───test - ymls for testing the networks
│       └───train - ymls for training the networks
│       │   options.py
│   └───scripts
│   └───utils
|   calc_psnr_ssim.py - code for calculating psnrs and ssims for image sets
|   degradation_gen.py - code for generating the preset kernel
|   make_downscaled_images.py - code for making true slr images
|   make_slr_images.py - code for making slr images generated by MFDN, SFDN
|   train.py - code for training the VSR network
|   train_dynavsr.py  - code for training DynaVSR
|   train_mfdn.py - code for training MFDN, SFDN network
|   test_maml.py - code for testing DynaVSR
|   test_Vid4_REDS4_with_GT(_DUF, _TOF).py - code for testing baseline VSR network
|   run_downscaling.sh - scripts for generating LR, SLR images
|   run_visual.sh - scripts for testing DynaVSR
| ...

Dependencies

Current version is tested on:

• Ubuntu 18.04
• Python==3.7.7
• numpy==1.17
• PyTorch==1.3.1, torchvision==0.4.2, cudatoolkit==10.0
• tensorboard==1.14.0
• etc: pyyaml, opencv, scikit-image, pandas, imageio, tqdm

# Easy installation (using Anaconda environment)
conda env create -f environment.yml
conda activate dynavsr

Model

Dataset Preparation

• Vimeo90K: Training / Vid4: Validation
• REDS: Training, Validation
  • download train_sharp data
• after downloading the dataset, use run_downscaling.sh to make lr, slr images
  • sh codes/run_downscaling.sh
• make symbolic link to the datasets.

Downloading Pretrained Models

• You can download pretrained models using the link: [Google Drive]

Usage

• To run EDVR, first install Deformable Convolution. We use mmdetection's dcn implementation. Please first compile it.

  cd ./codes/models/archs/dcn
  python setup.py develop
  

Training

Two ways to train DynaVSR network.

• Distributed training(When using multiple GPUs).

  cd ./codes
  python -m torch.distributed.launch --nproc_per_node=8 --master_port=4321 train_dynavsr.py -opt options/train/[Path to YML file] --launcher pytorch --exp_name [Experiment Name]
  

• Single GPU training.

  cd ./codes
  python train_dynavsr.py -opt options/train/[Path to YML file] --exp_name [Experiment Name]
  

Testing

• We just support single GPU for testing.

  • To use your own test set, change the dataroot in EDVR_Demo.yml file to the folder containing images.

  cd ./codes
  python test_dynavsr.py -opt options/test/[Path to YML file]
  

• Or just use run_visual.sh (Uncomment line that you want to execute).

  sh ./codes/run_visual.sh
  

Results

References

If you like the paper, please cite our paper using:

@inproceedings{lee2021dynavsr,
  title={DynaVSR: Dynamic Adaptive Blind Video Super-Resolution},
  author={Lee, Suyoung and Choi, Myungsub and Lee, Kyoung Mu},
  booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
  pages={2093--2102},
  year={2021}
}

Update Log

• [2020-12-09] First code cleanup, Publish pretrained models
• [2020-12-29] Add arbitrary input data testing script
• [2021-01-05] Add script for testing 4X scale EDVR models
• [2021-01-06] Modify scripts for generating LR, SLR frames

Acknowledgement

The code is built based on

• EDVR-Pytorch