Self-supervised Video Object Segmentation by Motion Grouping
This code accompanies the paper: Self-supervised Video Object Segmentation by Motion Grouping
Charig Yang, Hala Lamdouar, Erika Lu, Andrew Zisserman, Weidi Xie.
ICCV 2021
Project page: https://charigyang.github.io/motiongroup/
Requirements :
pytorch (tested on 1.7, although any recent version should work)
cvbase
einops
tensorboardX
Datasets :
- DAVIS 2016 can be used as-is.
- The rest has to be converted to DAVIS format. Some helper functions are available in tools.
- MoCA needs to be processed. See Supplementary Material for the paper for details. Helper functions are available in tools. The (already filtered) dataset is also available on google drive: https://drive.google.com/drive/u/2/folders/1x-owzr9Voz65NQghrN_H1LEYDaaQP5n1, which can be used as-is after download.
- Precomputed flows can be generated from raft/run_inference.py
Training :
python train.py --dataset DAVIS --flow_to_rgb
Inference :
python eval.py --dataset DAVIS --flow_to_rgb --inference --resume_path {}
Benchmark :
- For DAVIS: use the official evaluation code: https://github.com/fperazzi/davis
- For MOCA: use tools/MoCA_eval.py
How to use this on your own data :
- Generate optical flow from your dataset using raft/inference.py
- Edit setup_dataset in config.py to include your dataset, and add this to the choices in parser.add_argument('--dataset') in train.py and eval.py
- Follow the training and inference instructions above with your own --dataset argument. Use --resume_path {} if you are fine-tuning.
Issues/questions/pull requests :
are very welcome.
Reference :
If you find this helpful in your research, we would be grateful if you cite our work
@InProceedings{yang2021selfsupervised,
title={Self-supervised Video Object Segmentation by Motion Grouping},
author={Charig Yang and Hala Lamdouar and Erika Lu and Andrew Zisserman and Weidi Xie},
booktitle={ICCV},
year={2021},
}