TMO
This is the official PyTorch implementation of our paper:
Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation, WACV 2023
Suhwan Cho, Minhyeok Lee, Seunghoon Lee, Chaewon Park, Donghyeong Kim, Sangyoun Lee
URL: [Official] [arXiv]
PDF: [Official] [arXiv]
@article{TMO,
title={Treating Motion as Option to Reduce Motion Dependency in Unsupervised Video Object Segmentation},
author={Cho, Suhwan and Lee, Minhyeok and Lee, Seunghoon and Park, Chaewon and Kim, Donghyeong and Lee, Sangyoun},
booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
pages={5140--5149},
year={2023}
}
You can also find other related papers at awesome-video-object-segmentation.
Abstract
In unsupervised VOS, most state-of-the-art methods leverage motion cues obtained from optical flow maps in addition to appearance cues. However, as they are overly dependent on motion cues, which may be unreliable in some cases, they cannot achieve stable prediction. To overcome this limitation, we design a novel network that operates regardless of motion availability, termed as a motion-as-option network. Additionally, to fully exploit the property of the proposed network that motion is not always required, we introduce a collaborative network learning strategy. As motion is treated as option, fine and accurate segmentation masks can be consistently generated even when the quality of the flow maps is low.
Preparation
1. Download DUTS for network training.
2. Download DAVIS for network training and testing.
3. Download FBMS for network testing.
4. Download YouTube-Objects for network testing.
5. Save optical flow maps from DAVIS, FBMS, and YouTube-Objects using RAFT.
6. For convenience, I also provide the pre-processed DUTS, DAVIS, FBMS, and YouTube-Objects.
7. Replace dataset paths in "run.py" file with your dataset paths.
Training
1. Check training datasets in "run.py" file.
2. Run TMO training!!
python run.py --train
Testing
1. Make sure the pre-trained models are in your "trained_model" folder.
2. Select a pre-trained model and testing datasets in "run.py" file.
3. Run TMO testing!!
python run.py --test
4. You can also directly download pre-trained model and pre-computed results.
Note
Code and models are only available for non-commercial research purposes.
If you have any questions, please feel free to contact me :)
E-mail: chosuhwan@yonsei.ac.kr