Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

Yansong Tang *, Zhenyu Jiang *, Zhenda Xie *, Yue Cao, Zheng Zhang, Philip H. S. Torr, Han Hu (* equal contribution)

Introduction

This is the the repository for Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning, published in SRVU - ICCV 2021 workshop.

If you find our work useful in your research, please consider citing us.

@article{tang2021breaking,
  title={Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning},
  author={Tang, Yansong and Jiang, Zhenyu and Xie, Zhenda and Cao, Yue and Zhang, Zheng and Torr, Philip HS and Hu, Han},
  journal={arXiv preprint arXiv:2105.05838},
  year={2021}
}

Installation

Create a conda environment with Python 3.8.
Install Pytorch 1.5 with CUDA 10.2.
Install packages list in requirements.txt.
Install NVIDIA Apex following the instruction here.

Data

We use the Kinetics400 dataset. You can find directions for downloading it here.

To facilitates data preparation, we save the precomputed metadata given by torchvision.datasets.Kinetics400, and load it before training.

Training and Testing

Training

Run:

python -m torch.distributed.launch --nproc_per_node=$NUM_GPUS train.py -opt $OPTION_FILE_NAME -extra amp_opt_level=O1

An example option file is here

Testing

You could download our pretrained model here

We follow the CRW to perform downstream task evaluation

An example command is:

bash davis_test_script.sh $TRAINED_MODEL_PATH reproduce 0 -1

Steve-Tod / STFC3