JingchunCheng / Seg-with-SPN

Demo code of the paper: "Learning to Segment Instances in Videos with Spatial Propagation Network", in CVPR'17 Workshop on DAVIS Challenge

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Learning to Segment Instances in Videos with Spatial Propagation Network

alt text

This paper is available at the 2017 DAVIS Challenge website.

Check our results in this video.

Contact: Jingchun Cheng (chengjingchun at gmail dot com)

Cite the Paper

If you find that our method is useful in your research, please cite:

@article{DAVIS2017-6th,
  author = {J. Cheng and S. Liu and Y.-H. Tsai and W.-C. Hung and S. Gupta and J. Gu and J. Kautz and S. Wang and M.-H. Yang}, 
  title = {Learning to Segment Instances in Videos with Spatial Propagation Network}, 
  journal = {The 2017 DAVIS Challenge on Video Object Segmentation - CVPR Workshops}, 
  year = {2017}
}

About the Code

  • The code released here mainly consistes of two parts in the paper: foreground segmentation and instance recognition.

  • It contains the parent net for foreground segmentation and training codes for instance recognition networks.

  • The matlab_code folder contains a simple version of our CRAF step for segmentation refinement.

Requirements

Training

  • Train the per-object recognition model.
    cd training
    python solve.py PATH_OF_MODEL PATH_OF_SOLVER
    Foe example, on the 'choreography' video for the 1st object, run:
    python solve.py ../pretrained/PN_ResNetF.caffemodel ../ResNetF/testnet_per_obj/choreography/solver_1.prototxt

Testing

  • Test the general foreground/backgroung model.
    python infer_test_fgbg.py PATH_OF_MODEL PATH_OF_RESULT VIDEO_NAME
    Foe example, on the 'lions' video, run:
    python infer_test_fgbg.py pretrained/PN_ResNetF.caffemodel results/fgbg lions

  • Test the object instance model.
    python infer_test_perobj.py MODEL_ITERATION VIDEO_NAME OBJECT_ID
    For example, on the 'lions' video for the 2nd object, run:
    python infer_test_perobj.py 3000 lions 2

  • Run example_CRAF.m in the matlab_code folder for a demo on CRAF segmentation refinement.

Download Our Segmentation Results on 2017 DAVIS Challenge

  • General foreground/background segmentation here
  • Instance-level object segmentation without refinement here
  • Final instance-level object segmentation with refinement here

Note

The model and code are available for non-commercial research purposes only.

  • 09/2017: code and model released
  • 03/2018: pre-trained model updated

About

Demo code of the paper: "Learning to Segment Instances in Videos with Spatial Propagation Network", in CVPR'17 Workshop on DAVIS Challenge


Languages

Language:Python 88.1%Language:MATLAB 11.9%