zejizhu / Deep-Flow-Guided-Video-Inpainting

pytorch implementation for "Deep Flow-Guided Video Inpainting"(CVPR'19)

Home Page:https://nbei.github.io/video-inpainting.html

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Deep Flow-Guided Video Inpainting

CVPR 2019 Paper | Project Page | YouTube | BibeTex

Install & Requirements

The code has been tested on pytorch=0.4.0 and python3.6. Please refer to requirements.txt for detailed information.

To Install python packages

pip install -r requirements.txt

To Install flownet2 modules

bash install_scripts.sh

Componets

There exist three components in this repo:

Usage

  • To use our video inpainting tool for object removing, we recommend that the frames should be put into xxx/video_name/frames and the mask of each frame should be put into xxx/video_name/masks. And please download the resources of the demo and model weights from here. An example demo containing frames and masks has been put into the demo and running the following command will get the result:
python tools/video_inpaint.py --frame_dir ./demo/frames --MASK_ROOT ./demo/masks --img_size 512 832 --FlowNet2 --DFC --ResNet101 --Propagation 

We provide the original model weight used in our movie demo which use ResNet101 as backbone and other related weights pls download from here. Please refer to tools for detailed use and training settings.

  • For fixed region inpainting, we provide the model weights of refined stages in DAVIS. Please download the lady-running resources link and model weights link. The following command can help you to get the result:
CUDA_VISIBLE_DEVICES=0 python tools/video_inpaint.py --frame_dir ./demo/lady-running/frames \
--MASK_ROOT ./demo/lady-running/mask_bbox.png \
--img_size 448 896 --DFC --FlowNet2 --Propagation \
--PRETRAINED_MODEL_1 ./pretrained_models/resnet50_stage1.pth \
--PRETRAINED_MODEL_2 ./pretrained_models/DAVIS_model/davis_stage2.pth \
--PRETRAINED_MODEL_3 ./pretrained_models/DAVIS_model/davis_stage3.pth \
--MS --th_warp 3 --FIX_MASK

You can just change the **th_warp** param for getting better results in your video.
  • To extract flow for videos:
python tools/infer_flownet2.py --frame_dir xxx/video_name/frames
  • To use the Deepfillv1-Pytorch model for image inpainting,
python tools/frame_inpaint.py --test_img xxx.png --test_mask xxx.png --image_shape 512 512

Update

  • The frames and masks of our movie demo have been put into Google Drive.
  • The weights of DAVIS's refined stages have been released and you can download from here. Please refer to Usage for using the Multi-Scale models.

FAQ

  • Errors when running install_scripts.sh if you meet some problem about gcc when compiling, pls check if the following commands will help:
export CXXFLAGS="-std=c++11"
export CFLAGS="-std=c99"

Citation

@InProceedings{Xu_2019_CVPR,
author = {Xu, Rui and Li, Xiaoxiao and Zhou, Bolei and Loy, Chen Change},
title = {Deep Flow-Guided Video Inpainting},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}

About

pytorch implementation for "Deep Flow-Guided Video Inpainting"(CVPR'19)

https://nbei.github.io/video-inpainting.html

License:MIT License


Languages

Language:Python 74.9%Language:Cuda 16.1%Language:C 7.1%Language:C++ 1.4%Language:Shell 0.5%