Towards Efficient 3D Object Detection with Knowledge Distillation
Jihan Yang, Shaoshuai Shi, Runyu Ding, Zhe Wang, Xiaojuan Qi
This repository contains the official implementation of Towards Efficient 3D Object Detection with Knowledge Distillation, NeurIPS 2022
Changelog
[2022-09-] Code release for Waymo results.
Introduction
Our code is based on OpenPCDet v0.5.2. More updates on OpenPCDet are supposed to be compatible with our code.
Model Zoo
Waymo Open Dataset
Basic Models without KD
Similar to OpenPCDet, all models in the following table are trained with a single frame of 20% data (~32k frames) of all
the training samples on 8 GTX 1080Ti GPUs.
Note that the validation are also carried on the 20% validation set with DATA_CONFIG.SAMPLED_INTERVAL.test 5
(similar performance with 100% validation set).
| Model (20% data) | LEVEL2 mAPH | Flops (G) | Acts (M) | Latency (ms) |
|---|---|---|---|---|
| CP-Voxel | 64.29 | 114.8 | 101.9 | 125.70 |
| CP-Voxel-S | 62.23 | 47.8 | 65.7 | 97.99 |
| CP-Voxel-XS | 61.14 | 36.9 | 58.4 | 88.19 |
| CP-Voxel-XXS | 56.26 | 12.0 | 33.1 | 70.44 |
| CP-Pillar | 59.09 | 333.9 | 303.0 | 157.90 |
| CP-Pillar-v0.4 | 57.55 | 212.9 | 197.7 | 103.37 |
| CP-Pillar-v0.48 | 56.27 | 149.4 | 142.3 | 81.87 |
| CP-Pillar-v0.64 | 52.81 | 85.1 | 88.0 | 54.52 |
Sparse Distillation Models
| Model (20% data) | LEVEL2 mAPH | Gains |
|---|---|---|
| CP-Voxel-S + SparseKD | 64.25 | +2.0 |
| CP-Voxel-XS + SparseKD | 63.53 | +2.4 |
| CP-Voxel-XXS + SparseKD | 59.28 | +3.0 |
| CP-Pillar-v0.4 + SparseKD | 59.24 | +1.7 |
| CP-Pillar-v0.48 + SparseKD | 58.53 | +2.3 |
| CP-Pillar-v0.64 + SparseKD | 55.82 | +3.0 |
Here we also provide the performance of several models trained on the full training set and validate on the full validation set.
| Model (100% data) | LEVEL2 mAPH |
|---|---|
| CP-Voxel | 65.58 |
| CP-Pillar | 61.56 |
| PV-RCNN++ | 69.46 |
| CP-Voxel-S + SparseKD | 65.75 |
| CP-Voxel-XS + SparseKD | 64.83 |
| CP-Voxel-XXS + SparseKD | 60.93 |
| CP-Pillar-v0.4 + SparseKD | 61.60 |
| CP-Pillar-v0.48 + SparseKD | 60.95 |
| CP-Pillar-v0.64 + SparseKD | 58.89 |
Cross stage distillation
| Model (20% data) | LEVEL2 mAPH | Flops (G) | Acts (M) | Latency (ms) |
|---|---|---|---|---|
| PV-RCNN++ | 67.80 | 123.5 | 179.7 | 435.9 |
| CP-Voxel | 64.29 | 114.8 | 101.9 | 125.7 |
| CP-Voxel + SparseKd | 65.27 | 114.8 | 101.9 | 125.7 |
We could not publicly provide the above pretrained models due to Waymo Dataset License Agreement. To access these pretrained models, please email us your name, institute, a screenshot of the Waymo dataset registration confirmation mail, and your intended usage. Please send a second email if we don't get back to you in two days. Please note that Waymo open dataset is under strict non-commercial license, so we are not allowed to share the model with you if it will use for any profit-oriented activities.
Latency on Different Hardware
To facilitate further researches, we provide latency measured in milliseconds as follows:
| Model | 1060 + Spconv2.x | 1080Ti + Spconv2.x | A100 + Spconv2.x |
|---|---|---|---|
| SECOND | 84.56 | 50.65 | 40.44 |
| PointPillar | 129.12 | 68.61 | 44.14 |
| CP-Voxel | 125.70 | 74.25 | 56.13 |
| CP-Voxel-S | 97.99 | 65.68 | 39.40 |
| CP-Voxel-XS | 88.19 | 61.41 | 36.65 |
| CP-Voxel-XXS | 70.44 | 53.86 | 38.01 |
| CP-Pillar | 157.90 | 77.29 | 51.94 |
| CP-Pillar-v0.4 | 103.37 | 56.32 | 57.07 |
| CP-Pillar-v0.48 | 81.87 | 49.64 | 27.44 |
| CP-Pillar-v0.64 | 54.52 | 36.23 | 19.56 |
Installation
Please refer to INSTALL.md for the installation of OpenPCDet.
Getting Started
Please refer to GETTING_STARTED.md to learn more usage about this project.
License
Our code is released under the Apache 2.0 license.
Acknowledgement
Our code is heavily based on OpenPCDet. Thanks OpenPCDet Development Team for their awesome codebase.
Citation
If you find this project useful in your research, please consider cite:
@inproceedings{yang2022towards,
title={Towards Efficient 3D Object Detection with Knowledge Distillation},
author={Yang, Jihan and Shi, Shaoshuai and Ding, Runyu and Wang, Zhe and Qi, Xiaojuan},
booktitle={Advances in Neural Information Processing Systems},
year={2022}
}
@misc{openpcdet2020,
title={OpenPCDet: An Open-source Toolbox for 3D Object Detection from Point Clouds},
author={OpenPCDet Development Team},
howpublished = {\url{https://github.com/open-mmlab/OpenPCDet}},
year={2020}
}