mcahny / Deep-Video-Inpainting

Official pytorch implementation for "Deep Video Inpainting" (CVPR 2019)

Deep_Video_Inpainting

Official pytorch implementation for "Deep Video Inpainting" (CVPR 2019, TPAMI 2020)
Dahun Kim*, Sanghyun Woo*, Joon-Young Lee, and In So Kweon. (*: equal contribution)
[Paper] [Project page] [Video results]

If you are also interested in video caption removal, please check [Paper] [Project page]

Update

This is recently tested under Python=3.7, Pytorch=1.4, Cudatoolkit=10.0.

Disclaimer

This is tested under Python=3.6, PyTorch=0.4.0
Python=3.7, Pytorch=1.4, Cudatoolkit=10.0.

Testing

0. Download the trained weight 'save_agg_rec_512.pth' and place it in "./results/vinet_agg_rec/"
   Google drive: [weight-512x512] [weight-256x256]

1. Create conda env and install Pytorch library.

conda create -n vinet python=3.7 -y
conda activate vinet
conda install pytorch=1.4 torchvision cudatoolkit=10.0 -c pytorch -y

2. Compile Resample2d, Correlation dependencies.

bash ./install.sh

3. Run the demo (the results are saved in "./results/vinet_agg_rec/davis_512/").

python demo_vi.py

4. Optional: Run the video retargeting (Section 4.5)

python demo_retarget.py

Citation

If you find the codes useful in your research, please cite:

@inproceedings{kim2019deep,
  title={Deep Video Inpainting},
  author={Kim, Dahun and Woo, Sanghyun and Lee, Joon-Young and Kweon, In So},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={5792--5801},
  year={2019},}

@ARTICLE{kim2020vipami,
  author={Kim, Dahun and Woo, Sanghyun and Lee, Joon-Young and Kweon, In So},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  title={Recurrent Temporal Aggregation Framework for Deep Video Inpainting},
  year={2020},
  volume={42},
  number={5},
  pages={1038-1052},}