xvjiarui/GSPN

GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud

Created by Li (Eric) Yi, Wang Zhao, He Wang, Minhyuk Sung, Leonidas J. Guibas.

Citation

If you find our work useful in your research, please consider citing:

    @article{yi2018gspn,
      title={Gspn: Generative shape proposal network for 3d instance segmentation in point cloud},
      author={Yi, Li and Zhao, Wang and Wang, He and Sung, Minhyuk and Guibas, Leonidas},
      journal={arXiv preprint arXiv:1812.03320},
      year={2018}
    }

Introduction

This work is based on our CVPR'19 paper. You can find arXiv version of the paper here. We introduce a 3D object proposal approach named Generative Shape Proposal Network (GSPN) for instance segmentation in point cloud data. We incorporate GSPN into a novel 3D instance segmentation framework named Region-based PointNet (R-PointNet) which allows flexible proposal refinement and instance segmentation generation.

In this repository we release code and pre-trained model for both GSPN and R-PointNet.

Usage

We provide a step-by-step usage instruction from data processing to network evaluation on the ScanNet dataset. Please download ScanNet data and store in the "data" folder first.

Compiling the TF operators
```
 cd tf_ops
 . ./tf_all_compile.sh
```
Data pre-processing: convert ScanNet into downsampled point cloud for fast training and evaluation
```
 python data_prep.py ./data/scannet ./data/scannet_preprocessed
 python dataset.py
```

Two-step training: we train GSPN and R-PointNet separately in two different stages

 python train.py --train_module SPN --log_dir log_spn
 python train.py --train_module RPOINTNET --log_dir log_rpointnet --restore_model_path log_spn/model.ckpt --restore_scope shape_proposal_net

Evaluation on the validation set: we evaluation on downsampled point cloud from ScanNet validation scenes, where we first generate predictions and then evaluate based upon the official code provided by ScanNet Benchmark

 python test.py --model_path log_rpointnet/model.ckpt
 cd ./external/ScanNet/BenchmarkScripts/3d_evaluation
 python evaluate_semantic_instance.py --pred_path root_dir/eva/pred --gt_path root_dir/eva/gt --output_file root_dir/eva/pred/semantic_instance_evaluation.txt

Pre-trained model

The pretrained R-PointNet could be downloaded here (57Mb).

License

Our code is released under MIT License (see LICENSE file for details).

xvjiarui / GSPN