TianhaoFu / Awesome-3D-Object-Detection

Papers, code and datasets about deep learning for 3D Object Detection.

Awesome-3D-Object-Detection

A curated list of research in 3D Object Detection(Lidar-based Method).

You are very welcome to pull request to update this list. 😃

Dataset

• KITTI Dataset

  • 3,712 training samples
  • 3,769 validation samples
  • 7,518 testing samples

• nuScenes Dataset

  • 28k training samples
  • 6k validation samples
  • 6k testing samples

• Lyft Dataset

• Waymo Open Dataset

  • 798 training sequences with around 158, 361 LiDAR samples
  • 202 validation sequences with 40, 077 LiDAR samples.

Top conference & workshop

Conferene

• Conference on Computer Vision and Pattern Recognition(CVPR)
• International Conference on Computer Vision(ICCV)
• European Conference on Computer Vision(ECCV)

Workshop

• CVPR 2019 Workshop on Autonomous Driving(nuScenes 3D detection)

• CVPR 2020 Workshop on Autonomous Driving(BDD1k 3D tracking)

• CVPR 2021 Workshop on Autonomous Driving(waymo 3D detection)

• CVPR 2022 Workshop on Autonomous Driving(waymo 3D detection)

• CVPR 2021 Workshop on 3D Vision and Robotics

• CVPR 2021 Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics

• ICCV 2019 Workshop on Autonomous Driving

• ICCV 2021 Workshop on Autonomous Vehicle Vision (AVVision), note

• ICCV 2021 Workshop SSLAD Track 2 - 3D Object Detection

• ECCV 2020 Workshop on Commands for Autonomous Vehicles

• ECCV 2020 Workshop on Perception for Autonomous Driving

Paper (Lidar-based method)

• End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Clouds paper
• Vehicle Detection from 3D Lidar Using Fully Convolutional Network(baidu) paper
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection paper
• Object Detection and Classification in Occupancy Grid Maps using Deep Convolutional Networks paper
• RT3D: Real-Time 3-D Vehicle Detection in LiDAR Point Cloud for Autonomous Driving paper
• BirdNet: a 3D Object Detection Framework from LiDAR information paper
• LMNet: Real-time Multiclass Object Detection on CPU using 3D LiDAR paper
• HDNET: Exploit HD Maps for 3D Object Detection paper
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation paper
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space paper
• IPOD: Intensive Point-based Object Detector for Point Cloud paper
• PIXOR: Real-time 3D Object Detection from Point Clouds paper
• DepthCN: Vehicle Detection Using 3D-LIDAR and ConvNet paper
• Voxel-FPN: multi-scale voxel feature aggregation in 3D object detection from point clouds paper
• STD: Sparse-to-Dense 3D Object Detector for Point Cloud paper
• Fast Point R-CNN paper
• StarNet: Targeted Computation for Object Detection in Point Clouds paper
• Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection paper
• LaserNet: An Efficient Probabilistic 3D Object Detector for Autonomous Driving paper
• FVNet: 3D Front-View Proposal Generation for Real-Time Object Detection from Point Clouds paper
• Part-A^2 Net: 3D Part-Aware and Aggregation Neural Network for Object Detection from Point Cloud paper
• PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud paper
• Complex-YOLO: Real-time 3D Object Detection on Point Clouds paper
• YOLO4D: A ST Approach for RT Multi-object Detection and Classification from LiDAR Point Clouds paper
• YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud paper
• Monocular 3D Object Detection with Pseudo-LiDAR Point Cloud paper
• Pillar-based Object Detection for Autonomous Driving (ECCV2020) paper
• EPNet: Enhancing Point Features with Image Semantics for 3D Object Detection(ECCV2020) paper
• Multi-Echo LiDAR for 3D Object Detection(ICCV2021) paper
• LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector(ICCV2021) paper
• SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation(ICCV2021) paper
• Structure Aware Single-stage 3D Object Detection from Point Cloud（CVPR2020) paper code
• MLCVNet: Multi-Level Context VoteNet for 3D Object Detection（CVPR2020) paper code
• 3DSSD: Point-based 3D Single Stage Object Detector（CVPR2020） paper code
• LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attention（CVPR2020） paper code
• PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection(CVPR2020) paper code
• Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud（CVPR2020） paper code
• MLCVNet: Multi-Level Context VoteNet for 3D Object Detection（CVPR2020） paper
• Density Based Clustering for 3D Object Detection in Point Clouds（CVPR2020） paper
• What You See is What You Get: Exploiting Visibility for 3D Object Detection（CVPR2020) paper
• PointPainting: Sequential Fusion for 3D Object Detection(CVPR2020) paper
• HVNet: Hybrid Voxel Network for LiDAR Based 3D Object Detection（CVPR2020) paper
• LiDAR R-CNN: An Efficient and Universal 3D Object Detector（CVPR2021) paper
• Center-based 3D Object Detection and Tracking(CVPR2021) paper
• 3DIoUMatch: Leveraging IoU Prediction for Semi-Supervised 3D Object Detection(CVPR2021) paper
• Embracing Single Stride 3D Object Detector with Sparse Transformer(CVPR2022) paper, code
• Point Density-Aware Voxels for LiDAR 3D Object Detection(CVPR2022) paper, code
• A Unified Query-based Paradigm for Point Cloud Understanding(CVPR2022) paper
• Beyond 3D Siamese Tracking: A Motion-Centric Paradigm for 3D Single Object Tracking in Point Clouds(CVPR2022) paper, code
• Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds(CVPR2022) paper, code
• Back To Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement(CVPR2022) paper, code
• Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Clouds(CVPR2022) paper, code
• BoxeR: Box-Attention for 2D and 3D Transformers(CVPR2022) paper, code, 中文介绍
• Canonical Voting: Towards Robust Oriented Bounding Box Detection in 3D Scenes(CVPR2022) paper, code
• DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detection(CVPR2022) paper, code
• TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers. (CVPR2022) paper, code
• Point2Seq: Detecting 3D Objects as Sequences. (CVPR2022) paper, code
• CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detection(CVPR2022) paper
• LiDAR Snowfall Simulation for Robust 3D Object Detection(CVPR2022) paper, code
• Unified Transformer Tracker for Object Tracking(CVPR2022) paper, code
• Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion(CVPR2022) paper
• M^2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation(CVPR2022) paper
• RBGNet: Ray-based Grouping for 3D Object Detection(CVPR2022) paper, code
• Fast Point Transformer(CVPR2022) paper
• Focal Sparse Convolutional Networks for 3D Object Detection(CVPR2022) paper, code
• FUTR3D: A Unified Sensor Fusion Framework for 3D Detection(CVPR2022) paper
• VISTA: Boosting 3D Object Detection via Dual Cross-VIew SpaTial Attention(CVPR2022) paper, code
• OccAM’s Laser: Occlusion-based Attribution Maps for 3D Object Detectors on LiDAR Data(CVPR2022) paper
• Voxel Field Fusion for 3D Object Detection(CVPR2022) paper, code
• FSD V2: Improving Fully Sparse 3D Object Detection with Virtual Voxels
• LinK: Linear Kernel for LiDAR-based 3D Perception(CVPR2023) paper, code
• DSVT: Dynamic Sparse Voxel Transformer with Rotated Sets(CVPR2023) paper, code
• VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking(CVPR2023) paper, code
• LargeKernel3D: Scaling up Kernels in 3D Sparse CNNs(CVPR2023) paper, code
• FocalFormer3D : Focusing on Hard Instance for 3D Object Detection(ICCV2023) paper, code
• CTRL: Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection(ICCV2023) paper, code
• Real-Aug: Realistic Scene Synthesis for LiDAR Augmentation in 3D Object Detection(arxiv2023) paper, code

Competition Solution

• 6th AI Driving Olympics, ICRA 2021
• 5th AI Driving Olympics, NeurIPS 2020
• Workshop on Benchmarking Progress in Autonomous Driving, ICRA 2020
• Workshop on Autonomous Driving, CVPR 2019

Engineering

• Pointpillars-ONNX code
• Centerpoint-ONNX code
• BEVFormer-TensorRT code

Survey

• 2021.04 Point-cloud based 3D object detection and classification methods for self-driving applications: A survey and taxonomy paper
• 2021.07 3D Object Detection for Autonomous Driving: A Survey paper
• 2021.07 Multi-Modal 3D Object Detection in Autonomous Driving: a Survey paper
• 2021.10 A comprehensive survey of LIDAR-based 3D object detection methods with deep learning for autonomous driving paper
• 2021.12 Deep Learning for 3D Point Clouds: A Survey paper

Book

• 3D Object Detection Algorithms Based on Lidar and Camera: Design and Simulation book

Video

• Aivia online workshop: 3D object detection and tracking video
• 3D Object Retrieval 2021 workshop video
• 3D Deep Learning Tutorial from SU lab at UCSD video
• Lecture: Self-Driving Cars (Prof. Andreas Geiger, University of Tübingen) video
• Current Approaches and Future Directions for Point Cloud Object (2021.04) video
• Latest 3D OBJECT DETECTION with 30+ FPS on CPU - MediaPipe and OpenCV Python (2021.05) video
• MIT autonomous driving seminar (2019.11) video
• sensetime seminar1 video
• sensetime seminar2 slides

Course

• University of Toronto, csc2541
• University of Tübingen, Self-Driving Cars (Strong Recommendation)
• baidu-Udacity
• baidu-apollo
• University of Toronto, coursera

Blog

• Waymo Blog
• apollo介绍之Perception模块
• Apollo notes (Apollo学习笔记) - Apollo learning notes for beginners.
• PointNet系列论文解读
• Deep3dBox: 3D Bounding Box Estimation Using Deep Learning and Geometry
• SECOND算法解析
• PointRCNN深度解读
• Fast PointRCNN论文解读
• PointPillars论文和代码解析
• VoxelNet论文和代码解析
• CenterPoint源码分析
• PV-RCNN: 3D目标检测 Waymo挑战赛+KITTI榜 单模态第一算法
• LiDAR R-CNN：一种快速、通用的二阶段3D检测器
• 混合体素网络（HVNet)
• 自动驾驶感知| Range Image paper分享
• SST：单步长稀疏Transformer 3D物体检测器

Famous Research Group/Scholar

• Naiyan Wang@Tusimple
• Hongsheng Li@CUHK
• Oncel Tuzel@Apple
• Oscar Beijbom@nuTonomy
• Raquel Urtasun@University of Toronto
• Philipp Krähenbühl@UT Austin
• Deva Ramanan@CMU
• Jiaya Jia@CUHK
• Thomas Funkhouser@princeton
• Leonidas Guibas@Stanford
• Steven Waslander@University of Toronto
• Ouais Alsharif@Google Brain
• Yuning CHAI(former)@waymo
• Yulan Guo@NUDT
• Lei Zhang@The Hong Kong Polytechnic University
• Hongyang Li@sensetime
• Luc Van Gool@ETH
• Sanja Fidler@NVIDIA
• Alan L. Yuille@JHU
• OpenDriveLab

Famous CodeBase

• Point Cloud Library (PCL)
• Torchsparse
• Spconv
• Det3D
• mmdetection3d
• OpenPCDet
• Centerpoint
• Apollo Auto - Baidu open autonomous driving platform
• AutoWare - The University of Tokyo autonomous driving platform
• Openpilot - A open source software built to improve upon the existing driver assistance in most new cars on the road today
• DeepVision3D
• MinkowskiEngine

Famous Toolkit

• ZED Box

Acknowlegement

Awesome System for Machine Learning

awesome-3D-object-detection