zyf12389 / SegStereo

SegStereo: Exploiting Semantic Information for Disparity Estimation

SegStereo

Caffe implementation of SegStereo and ResNetCorr models.

Requirements

This code is tested with Caffe, CUDA 8.0 and Ubuntu 16.04.

• Basic caffe implementation is from Caffe.
• The correlation and correlation1d layers are from FlowNet 2.0.
• The Interp layer is from PSPNet.
• The disparity tool is from OpticalFlowToolkit

Data

Our models require rectified stereo pairs. We provide several examples in data directory

• KITTI Stereo 2012
• KITTI Stereo 2015
• FlyingThings3D
• Cityscapes

Models

• ResNetCorr_SRC_pretrain.caffemodel: Google Drive
• SegStereo_SRC_pretrain.caffemodel: Google Drive
• SegStereo_pre_corr_SRC_pretrain.caffemodel: Google Drive
• ResNetCorr_KITTI_finetune.caffemodel: Google Drive
• SegStereo_KITTI_finetune.caffemodel: Google Drive
• SegStereo_pre_corr_KITTI_finetune.caffemodel: Google Drive

Evaluation

To test or evaluate the disparity model, you can use the script in model/get_disp.py. We recommend that you put the model under correponding directory.

python get_disp.py --model_weights ./ResNetCorr/ResNetCorr_SRC_pretrain.caffemodel --model_deploy ./ResNetCorr/ResNetCorr_deploy.prototxt --data ../data/KITTI --result ./ResNetCorr/result/kitti --gpu 0 --colorize --evaluate

Reference

• If our SegStereo or ResNetCorr models help your research, please consider citing:

@inproceedings{yang2018SegStereo,
  author    = {Yang, Guorun and
               Zhao, Hengshuang and
               Shi, Jianping and
               Deng, Zhidong and
               Jia, Jiaya},
  title     = {SegStereo: Exploiting Semantic Information for Disparity Estimation},
  booktitle = ECCV,
  year      = {2018}
}

• If you find our synthetic realistic collaborative (SRC) training strategy useful, please consider citing:

@inproceedings{yang2018srcdisp,
  author    = {Yang, Guorun and
               Deng, Zhidong and
               Lu, Hongchao and
               Li, Zeping},
  title     = {SRC-Disp: Synthetic-Realistic Collaborative Disparity Learning for Stereo Mathcing},
  booktitle = ACCV,
  year      = {2018}
}

Questions

Please contact [e-mail]