Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

The source code of our work "Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation

News

2020-11 We preliminarily release the Cylinder3D--v0.1, supporting the LiDAR semantic segmentation on SemanticKITTI and nuScenes.
2020-11 Our work achieves the 1st place in the leaderboard of SemanticKITTI semantic segmentation (until CVPR2021 DDL, still rank 1st in term of Accuracy now), and based on the proposed method, we also achieve the 1st place in the leaderboard of SemanticKITTI panoptic segmentation.

Installation

Requirements

PyTorch >= 1.2 (pip install torch==1.6.0 torchvision==0.7.0)
yaml
Cython
[torch-scatter](https://github.com/rusty1s/pytorch_scatter, pip install torch-scatter)
nuScenes-devkit (optional for nuScenes)
spconv (tested with spconv==1.2.1 and cuda==10.2)

Data Preparation

SemanticKITTI

./
├── 
├── ...
└── path_to_data_shown_in_config/
    ├──sequences
        ├── 00/           
        │   ├── velodyne/	
        |   |	├── 000000.bin
        |   |	├── 000001.bin
        |   |	└── ...
        │   └── labels/ 
        |       ├── 000000.label
        |       ├── 000001.label
        |       └── ...
        ├── 08/ # for validation
        ├── 11/ # 11-21 for testing
        └── 21/
	    └── ...

nuScenes

./
├── 
├── ...
└── path_to_data_shown_in_config/
		├──v1.0-trainval
		├──v1.0-test
		├──samples
		├──sweeps
		├──maps

Training

modify the config/semantickitti.yaml with your custom settings. We provide a sample yaml for SemanticKITTI.
train the network by running "sh train.sh".

Pretrained Models

-- We provide a pretrained model for SemanticKITTI LINK1 or LINK2 (access code: xqmi)

TODO List

Release pretrained model for nuScenes.
Support more models, including PolarNet, RandLA, SequeezeV3 and etc.
Support more datasets, including A2D2 and etc.
Integrate LiDAR Panoptic Segmentation into the codebase.

Reference

If you find our work useful in your research, please consider citing our paper:

@article{zhu2020cylindrical,
  title={Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation},
  author={Zhu, Xinge and Zhou, Hui and Wang, Tai and Hong, Fangzhou and Ma, Yuexin and Li, Wei and Li, Hongsheng and Lin, Dahua},
  journal={arXiv preprint arXiv:2011.10033},
  year={2020}
}

#for LiDAR panoptic segmentation
@article{hong2020lidar,
  title={LiDAR-based Panoptic Segmentation via Dynamic Shifting Network},
  author={Hong, Fangzhou and Zhou, Hui and Zhu, Xinge and Li, Hongsheng and Liu, Ziwei},
  journal={arXiv preprint arXiv:2011.11964},
  year={2020}
}

Acknowledgments

We thanks for the opensource codebases, PolarSeg, spconv and SPVNAS

Yvanali / Cylinder3D