HINTED: Hard Instance Enhanced Detector with Mixed-Density Feature Fusion for Sparsely-Supervised 3D Object Detection

This is a official code release of HINTED (Hard Instance Enhanced Detector with Mixed-Density Feature Fusion for Sparsely-Supervised 3D Object Detection). This code is mainly based on OpenPCDet.

Detection Framework

Getting Started

Installation

a. Clone this repository.

git clone https://github.com/xmuqimingxia/HINTED.git

b. Create virtual-env.

conda create -n HINTED python=3.8

b.1 cuda-11.4、cuda-11.6、cuda-11.7 tested

conda activate HINTED
pip install torch==1.13.0+cu116 torchvision==0.14.0+cu116 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu116
pip install spconv-cu116	
pip install -r requirements.txt
python setup.py develop

b.2 cuda-12.x not tested

conda activate HINTED
pip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu121
pip install spconv-cu120
pip install -r requirements.txt
python setup.py develop

Prepare dataset and pre-training model

pre_train model: CoIn(VoxelRCNN-based)

train_info: Sparsely-supervised train info

label_idx: label_idx.txt

Please download the official KITTI 3D object detection dataset and organize the downloaded files as follows (the road planes could be downloaded from [road plane], which are optional for data augmentation in the training):

HINTED
├── data
│   ├── kitti
│   │   │── ImageSets
│   │   │── training
│   │   │   ├──calib & velodyne & label_2 & image_2 & (optional: planes) & (optional: depth_2)
│   │   │── testing
│   │   │   ├──calib & velodyne & image_2
|   |   |── kitti_infos_train_coin.pkl
├── pcdet
├── tools

Generate the data infos by runing the following command:

bash remove.sh

Training

cd ../../tools

*if you use singe gpu run （--labeled_frame_idx is provideed above:label_idx.txt）

python train.py --cfg_file cfgs/kitti_models/voxel_rcnn_3classes_ssl_centerHead.yaml --pretrained_model <path_to_pretrained_model_CoIn(VoxelRCNN-based)> --labeled_frame_idx <path_to_label_idx.txt>

*if you use multi 8 gpus run

bash scripts/dist_train.sh 8 --cfg_file cfgs/kitti_models/voxel_rcnn_3classes_ssl_centerHead.yaml --pretrained_model <path_to_pretrained_model_CoIn(VoxelRCNN-based)> --labeled_frame_idx <path_to_label_idx.txt>

Citation

If you find this project useful in your research, please consider cite:

@inproceedings{hinted,
  title={HINTED: Hard Instance Enhanced Detector with Mixed-Density Feature Fusion for Sparsely-Supervised 3D Object Detection},
  author={Xia, Qiming and Ye, Wei and Wu, Hai and Zhao, Shijia and Xing, Leyuan and Huang, Xun and Deng, Jinhao and Li, Xin and Wen, Chenglu and Wang, Cheng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={15321--15330},
  year={2024}
}

Acknowledgement

This code is based on OpenPCDet,3DIoUMatch,HSSDA and COIN .

If you find some help for you, star is a good reward ^_^.

xmuqimingxia / HINTED