yuk6heo / IVOS-ATNet

Pytorch implementation of ECCV2020 paper, "Interactive Video Object Segmentation Using Global and Local Transfer Modules"

Interactive Video Object Segmentation Using Global and Local Transfer Modules

Yuk Heo, Yeong Jun Koh, Chang-Su Kim

[Project page] [arXiv]

Implementation of ECCV2020 paper, "Interactive Video Object Segmentation Using Global and Local Transfer Modules"

Codes in this github:

1. DAVIS2017 evaluation based on the DAVIS framework
2. DAVIS2016 real-world evaluation GUI

Prerequisite

• cuda 10.0
• python 3.6
• pytorch 1.2.0
• davisinteractive 1.0.4
• corrlation package of FlowNet2
• numpy, cv2, PtQt5, and other general libraries of python3

Directory Structure

• root/libs: library of utility files.

• root/networks : network codes.

  • correlation_package.zip : conserves GPU memory by appling the correlation package of FlowNet2.
  • deeplab: applies ASPP module in decoders. [original code]
  • atnet.py: consists A-Net and T-Net.
  • ltm_transfer.py: transfers previous segmentation with the local affinity of the local transfer module.

• root/config.py : configurations.

• root/eval_davis-framework.py : DAVIS2017 evaluation based on the DAVIS framework.

• root/eval_real-world.py : DAVIS2016 real-world evaluation GUI (to be released).

Instruction

DAVIS2017 evaluation based on the DAVIS framework

1. Edit config.py to set the directory of your DAVIS2017 dataset and the gpu ID.
2. Unzip and build corrlation package by

cd ./networks
unzip correlation_package.zip
cd correlation_package
rm -rf *_cuda.egg-info build dist __pycache__
python3 setup.py install --user

    If you have problems in this step, you can find more information in the repository.

3. Download our network parameters and place the file as root/ATNet-checkpoint.pth.
4. Run with python3 eval_davis-framework.py.

DAVIS2016 real-world evaluation GUI

Multi-object GUI (for DAVIS2017) is available at our github page, [GUI-IVOS]

Reference

Please cite our paper if the implementations are useful in your work:

@Inproceedings{
Yuk2020IVOS,
title={Interactive Video Object Segmentation Using Global and Local Transfer Modules},
author={Yuk Heo and Yeong Jun Koh and Chang-Su Kim},
booktitle={ECCV},
year={2020},
url={https://openreview.net/forum?id=bo_lWt_aA}
}