praeclarumjj3 / PCNet-C-Experiments

Code for my experiments with PCNet-C.

PCNet-C Experiments on COCOA Dataset

Contents

1. Overview
2. Setup Instructions
3. Experiments
4. Results

1. Overview

This repo contains the code for my experiments on content completion using the PCNet-C model proposed in Self-Supervised Scene De-occlusion.

2. Setup Instructions

• Clone the repo:

https://github.com/praeclarumjj3/PCNet-C-Experiments.git
cd PCNet-C-Experiments

• Install Pytorch and other dependencies:

pip3 install -r requirements.txt

• Install pycocotools:

pip3 install "git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI"

Dataset Preparation

• Download the MS-COCO 2014 images and unzip:

wget http://images.cocodataset.org/zips/train2014.zip
wget http://images.cocodataset.org/zips/val2014.zip

• Download the annotations and untar:

gdown https://drive.google.com/uc?id=0B8e3LNo7STslZURoTzhhMFpCelE
tar -xf annotations.tar.gz

• Unzip the files according to the following structure

PCNet-C-Experiments
├── data
│   ├── COCOA
│   │   ├── annotations
│   │   ├── train2014
│   │   ├── val2014

3. Experiments

Training

1. Download the pre-trained image inpainting model using partial convolution here to pretrains/partialconv.pth

2. Convert the model to accept 4 channel inputs.

   python tools/convert_pcnetc_pretrain.py

3. Train (taking COCOA for example).

   sh experiments/train.sh # you may have to set --nproc_per_node=#YOUR_GPUS
   

• In the plots below:
  • Rec Loss = λ₁ L_hole + λ₂ L_valid + λ₃ L_sty + λ₄ L_prc + λ₅ L_tv

    λ<sub>1</sub> = 6.0;
    λ<sub>2</sub> = 1.0;
    λ<sub>3</sub> = 250.0;
    λ<sub>4</sub> = 0.1;
    λ<sub>5</sub> = 0.1;
    

  • Adv G Loss = 0.1 * L_{gen_gan}
  • Adv D Loss = L_{gen_disc}

The model was trained for 45000 iterations.

Demo

Run the following command:

sh experiments/demo.sh

Using Pretrained Model

• Download released model from here, rename it to G_iter_0.pth.tar and put it the folder saved_checkpoints under experiments.

gdown https://drive.google.com/uc?id=1MFK3mvSsB_T6IhmunPHi0EUWnDv-nDTa

• Set the --load_iter parameter equal to 0 in demo.sh.

4. Results

Using my Trained model

Using Pretrained model

Acknowledgement

This repo borrows heavily from deocclusion.