[Re] - VCNet: A Robust Approach to Blind Image Inpainting
This repository is the re-production implementation of VCNet: A Robust Approach to Blind Image Inpainting by Yi Wang, Ying-Cong Chen, Xin Tao and Jiaya Jia in the scope of ML Reproducibility Challenge 2020.
Requirements
To install requirements:
pip install -r requirements.txt
To download datasets, please visit the corresponding web pages: FFHQ - CelebAMaskHQ - Places - ImageNet - graffiti - raindrop.
After downloading, extract each of them to corresponding folder that you created as:
mkdir datasets
cd datasets
mkdir <dataset_name>
Training
To train VCNet from the scratch in the paper, run this command:
python main.py --base_cfg config.yml
Evaluation
To evaluate VCNet on FFHQ with a particular test mode, run:
python main.py --base_cfg config.yml -t -w vcnet.pth --dataset "FFHQ" --test_mode 1
Modes:
- 1: Contaminant image
- 2: Random brush strokes with noise
- 3: Random brush strokes with colors
- 4: Real occlusions
- 5: Graffiti
- 6: Facades (problematic)
- 7: Words (problematic)
- 8: Face swapping (problematic)
Pre-trained Models
You can download pretrained models here:
Qualitative Results
From FFHQ-CelebAMaskHQ model (from left to right: the original - the contaminant - broken - gt mask - smoothed mask - predicted mask - output):
From FFHQ-ImageNet model (from left to right: the original - the contaminant - broken - gt mask - smoothed mask - predicted mask - output):
From Places-ImageNet model (from left to right: the original - the contaminant - broken - gt mask - smoothed mask - predicted mask - output):
Quantitative Results
FFHQ
| Models | BCE | PSNR | SSIM |
|---|---|---|---|
| Contextual Attention | 1.297 | 16.56 | 0.5509 |
| GMC | 0.766 | 20.06 | 0.6675 |
| Partial Conv. | 0.400 | 20.19 | 0.6795 |
| Gated Conv. | 0.660 | 17.16 | 0.5915 |
| VCN (original) | 0.400 | 20.94 | 0.6999 |
| VCN (ours) | 0.439 | 24.76 | 0.7026 |
Places2
| Models | BCE | PSNR | SSIM |
|---|---|---|---|
| Contextual Attention | 0.574 | 18.12 | 0.6018 |
| GMC | 0.312 | 20.38 | 0.6956 |
| Partial Conv. | 0.273 | 19.73 | 0.6682 |
| Gated Conv. | 0.504 | 18.42 | 0.6423 |
| VCN (original) | 0.253 | 20.54 | 0.6988 |
| VCN (ours) | 0.437 | 21.53 | 0.7070 |