An anchor-free single-stage LiDAR-based 3D object detector purely based on the range view representation.
This is the official implementation of RangeDet (ICCV 2021).
- Create range images on KITTI
- Add installation instructions
- Add data augmentation
- Add Pedestrian model setting
- Add full data training results
RangeDet:In Defense of Range View for LiDAR-based 3D Object Detection [arXiv] [CVF] [Supplementary materials]
Lue Fan*, Xuan Xiong*, Feng Wang, Naiyan Wang , Zhaoxiang Zhang
*Equal contribution
We propose a pure range-view-based framework - RangeDet, which includes range conditioned pyramid, Meta-Kernel, weighted non-maximum suppression, and a iou-awared classification loss to overcome a couple of challenges in range-view-based 3D detector and achieves comparable performance with state-of-the-art multi-view-based detectors.
datasets
create_range_image_roidb.py
rangedet
symbol
backbone
dla_backbone.py
meta_kernel.py
head
builder.py
loss.py
core
detection_metric.py
input.py
util_func.py
config
...
scripts
distributed_train_and_test.sh
tools
train.py
test.py
create_prediction_bin_3d.py
Some extra points and reminders:
operator_cxx/contrib
contain MXNet's Custom Op written in C++, which need to be compiled with MXNxt.operator_cxx/src_cxx
contain wnms Op written in C++.pybind_cxx
contain bbox assigner function written in C++.operator_py
contain MXNet's Custom Op written in Python.mxnext
is a foundational library, we have modified some functions in this library.utils
contain modular python files, most of them copy from SimpleDet.
Experiments were performed using uniformly sample 25% training data (∼40k frames)
Method | Train epoch | data size | w/wo aug | Overall LEVEL 1 3D-AP/APH on Vehicle (IoU=0.7) | Overall LEVEL 2 3D-AP/APH on Vehicle (IoU=0.7) |
---|---|---|---|---|---|
rangedet_wo_aug_4_18e | 18 | 1/4 | no | 67.2/66.6 | 58.6/58.1 |
We will further add the experiment results here, for now please refer to our overall performance on Waymo leaderboard in this link .
Installing and compiling rangedet in the following command.
pip install -v -e .
We will further add requirements and installation instructions on how to install Horovod , OpenMPI and MXNet.
To train on the Waymo Open Dataset:
- Download the Waymo Open Dataset
- Unzip all
*.tar
files and merge them into three directories - Make sure to put the files as the following structure:
datasets
waymo
training/*.tfrecord
validation/*.tfrecord
testing/*.tfrecord
- Next you need to extract the data from tfrecord; We provide the python script in
datasets/create_range_image_roidb.py
- Store the training data in
npz
format and the label information inpickle
format respectively
To run the python script datasets/create_range_image_roidb.py
, do the following settings in the file:
python datasets/create_range_image_roidb.py --data_path "**/datasets/waymo" --save_path "**/datasets/waymo-range" --dataset-split 'training' --save_dir 'npz_data'
- After that, make sure to put the files as the following structure:
datasets
waymo-range
training/*.roidb
validation/*.roidb
testing/*.roidb
npz_data/**/*.npz
- Note: you need set
DatasetParam.data_root = 'path/to/datasets/waymo-range'
in theconfig/*.py
file.
Most of the configuration files that we provide assume that we are running on 8 GPUs. In order to be able to run it on fewer GPUs, there are a few possibilities:
1. Run the following
python tools/train.py --config "/path/to/config/file.py"
./scripts/horovodrun.sh 8 "python tools/train.py --config /path/to/config/file.py"
Note you don't need to change any config between single GPU training and multi-GPU training.
To enable, just do Single-GPU or Multi-GPU training and set General.fp16 = True
in the config/*.py
file.
We use Hough Transformation to obtain scanning parameters following RCD. Based on the scanning parameters, it easy to create high-quality range images.
To create range images on KITTI, just simply run:
python ./datasets/create_range_image_in_kitti.py --source-dir your_source_kitti_folder --target-dir your_save_directory
We assume that the source KITTI data is in MMDetection3D format. In other words, the folder your_source_kitti_path
is supposed to contains kitti_infos_trainval.pkl
and kitti_infos_test.pkl
. It is easy to create these files following the instruction of MMDetection3D.
You can test your model on single gpus. Here is an example:
python tools/test.py --config "/path/to/config/file.py"
You need to create a bin file so that you can use the official waymo evaluation metrics code to get the evaluation results.
python tools/create_prediction_bin_3d.py --exp_path "/path/to/experiments" --config "config_file" -epoch num_epoch --save_bin_path "/path/to/save_bin_file"
This project is based on SimpleDet. Thanks Yuntao Chen and his colleagues for their great work!
Please consider citing our paper in your publications if it helps your research.
@InProceedings{Fan_2021_ICCV,
author = {Fan, Lue and Xiong, Xuan and Wang, Feng and Wang, Naiyan and Zhang, ZhaoXiang},
title = {RangeDet: In Defense of Range View for LiDAR-Based 3D Object Detection},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
}
RangeDet is released under the Apache license. See LICENSE for additional details.