lharri73 / GaussMap

3D object detection algorithm for nuScenes dataset using 2d gaussian distributions. Designed for use with data from an Intel Mobileye; for this work, camera detections are generated using CenterTrack with a high confidence filter.

Gauss Map

A 3D object detector for the NuScenes dataset. Designed to work with a hardware monocular object detector and radar detections. In this work, camera detection points are generated with the CenterTrack algorithm, but this could be done using any 3D object detector.

Requirements

• cmake >=3.17
• pybind11 >=2.2 (installed with apt is easier)
• CUDA (tested on 10.2 & 11.2)

Installation

Installation is simplified with python. Simply run python setup.py install and dependencies will be installed and configured.

Getting Started

1. Follow the install steps above to setup the python library.

2. Run CenterTrack on all splits of the nuscenes dataset.

   These will be pickled together to allow for faster loading later. If you don't need all of them (for example, only the test for evaluation, then you can only run the test set)

   Here's the CenterTrack data I prepared before! note: It does not include the full dataset. Only the mini and test sets. Extract this to the results/CenterTrack directory.

3. From the root of this repository, run ./pySrc/main.py [nuscenes_version] [nuscenes_split] [nuscenes_root]

Running CenterTrack

A couple of changes have been made to centertrack to allow for it to run on NuScenes.

These changes can be found here.

This requires PyTorch version 1.4.0, torchvision 0.5.0, and Cuda <= 10.2

1. Follow the setup steps in the centertrack repository

   These are mostly in readme/INSTALL.md

2. Run the convert_nuscenes script. You'll have to adjust the DATA_ROOT at the top of this file

3. Create an alias in the root of your nuscenes directory called anotations that points to the directory containing the .json files just created.

4. Download the pretrained model called nuScenes_3Ddetection_e140 listed in readme/MODEL_ZOO.md. Place this in the models folder.

5. Run the following script, adjusting dataset_version for each split of the dataset you need

   python test.py ddd --exp_id nusc_det_full --load_model ../models/nuScenes_3Ddetection_e140.pth --dataset nuscenes --dataset_version mini-train

   and copy the .json file with the results in the exp/ddd/$exp_id$ folder to one of results/CenterTrack/{train,val,mini-train,mini-val,test}.