shelhamer / davis-interactive

Package to evaluate interactive segmentation with scribbles at DAVIS dataset.

DAVIS Interactive Evaluation Framework

This is a framework to evaluate interactive segmentation models over the DAVIS 2017 dataset. The code aims to provide an easy-to-use interface to test and validate interactive segmentation models.

This is the tool used to evaluate the interactive track on the DAVIS Challenge on Video Object Segmentation 2018. More info about the challenge on the official website.

You can find an example of how to use the package in the following repository:

• Scribble-OSVOS

DAVIS Scribbles

In the DAVIS Main Challenge track, the task consists on object segmentation in a semi-supervised manner, i.e. the given input is the ground truth mask of the first frame. In the DAVIS Interactive Challenge, in contrast, the user input is in form of scribbles, which can be drawn much faster by humans and thus is a more realistic type of input. The same objects as the ones of the Main track have been annotated with scribbles.

The interactive annotation and segmentation consist in an iterative loop which is going to be evaluated as follows:

• On the first iteration, a human-annotated scribble is provided to the segmentation model.
  Note: the annotated frame can be any of the sequence, as the annotators were instructed to annotate the most relevant and meaningful frame. This is in contrast to the Main track, where - only and strictly - the first frame is annotated.
• During the rest of the iterations, once the predicted masks have been submitted, a scribble is simulated by the server. The new annotation will be performed on a single frame and this frame will be chosen as the one on which the current result is the worst.

Evaluation: For now, the evaluation metric will be the Jaccard similarity $\mathcal{J}$.

Citation

Please cite both papers in your publications if DAVIS or this code helps your research.

@article{Caelles_arXiv_2018,
  author = {Sergi Caelles and Alberto Montes and Kevis-Kokitsi Maninis and Yuhua Chen and Luc {Van Gool} and Federico Perazzi and Jordi Pont-Tuset},
  title = {The 2018 DAVIS Challenge on Video Object Segmentation},
  journal = {arXiv:1803.00557},
  year = {2018}
}

@article{Pont-Tuset_arXiv_2017,
  author = {Jordi Pont-Tuset and Federico Perazzi and Sergi Caelles and Pablo Arbel\'aez and Alexander Sorkine-Hornung and Luc {Van Gool}},
  title = {The 2017 DAVIS Challenge on Video Object Segmentation},
  journal = {arXiv:1704.00675},
  year = {2017}
}