awesome-point-cloud-registration

A curated list of resources on point cloud registration inspired by awesome-computer-vision. Work-in-progress. All contributions are welcome and appreciated.

This list focuses on the rigid registration between point clouds.

Coarse Registration (Global Registration)
Fine Registration (Local Registration)
- Learning-based
- Traditional
Datasets
Tools

Coarse Registration

The coarse registration methods (or global registration) aligns two point clouds without an initial guess. We broadly classified these methods into feature matching based, end-to-end, randomized and probabilistic. Most of the learning based methods are focusing on some specific step in the feature matching based algorithms.

Feature Matching Based

The feature-matching based registration algorithms generally follow a two-stage workflow: determining correspondence and estimate the transformation. The correspondence establishing stage usually follow the four-step pipeline: keypoint detection, feature description, descriptor matching and outlier rejection. The nearest neighbor matching is the de-facto matching strategy, but could be replaced by learnable matching stategies. We also include some papers which adopt the graph algorithms for the matching and outlier rejection problem.

Keypoint Detection

HKS: A Concise and Provably Informative Multi‐Scale Signature Based on Heat Diffusion. CGF'2009 [paper]
Harris3D: a robust extension of the harris operator for interest point detection on 3D meshes. VC'2011 [paper]
Intrinsic shape signatures: A shape descriptor for 3D object recognition. ICCV'2012 [paper]
Learning a Descriptor-Specific 3D Keypoint Detector. ICCV'2015 [paper]
3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration. ECCV'2018 [paper] [code]
USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds. ICCV'2019 [paper] [code]
D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features. CVPR'2020 [paper] [code]
PointCloud Saliency Maps. ICCV'2019 [paper] [code]
SK-Net: Deep Learning on Point Cloud via End-to-end Discovery of Spatial Keypoints. AAAI'2020 [paper]
SKD: Unsupervised Keypoint Detecting for Point Clouds using Embedded Saliency Estimation. arxiv'2019 [paper]
Fuzzy Logic and Histogram of Normal Orientation-based 3D Keypoint Detection For Point Clouds. PRL'2020 [paper]
MaskNet: A Fully-Convolutional Network to Estimate Inlier Points. 3DV'2020 [paper] [code]
PREDATOR: Registration of 3D Point Clouds with Low Overlap. arxiv'2020 [paper] [code]

Survey:

Performance Evaluation of 3D Keypoint Detectors. IJCV'2013 [paper]

Feature Description

Spin Image: Using spin images for efficient object recognition in cluttered 3D scenes. TPAMI'1999 [paper]
USC: Unique shape context for 3D data description. 3DOR'2010 [paper]
3DShapeContext: Recognizing Objects in Range Data Using Regional Point Descriptors. ECCV'2004 [paper]
SHOT: Unique Signatures of Histograms for Local Surface Description. ECCV'2010 [paper]
FPFH: Fast Point Feature Histograms (FPFH) for 3D registration. ICRA'2009 [paper]
RoPS: 3D Free Form Object Recognition using Rotational Projection Statistics. WACV'2013 [paper]
CGF: Learning Compact Geometric Features. ICCV'2017 [paper]
3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions. CVPR'2017 [paper] [code]
End-to-end learning of keypoint detector and descriptor for pose invariant 3D matching. CVPR'2018 [paper]
PPFNet: Global Context Aware Local Features for Robust 3D Point Matching. CVPR'2018 [paper]
3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration. ECCV'2018 [paper] [code]
MVDesc: Learning and Matching Multi-View Descriptors for Registration of Point Clouds. ECCV'2018 [paper] [code]
FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation. CVPR'2018 [paper] [code]
PPF-FoldNet: Unsupervised Learning of Rotation Invariant 3D Local Descriptors. ECCV'2018 [paper] [code]
3D Local Features for Direct Pairwise Registration. CVPR'2019 [paper]
3D Point-Capsule Networks. CVPR'2019 [paper] [code]
The Perfect Match: 3D Point Cloud Matching with Smoothed Densities. CVPR'2019 [paper] [code]
FCGF: Fully Convolutional Geometric Features. ICCV'2019 [paper] [code]
Learning an Effective Equivariant 3D Descriptor Without Supervision. ICCV'2019 [paper]
D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features. CVPR'2020 [paper] [code]
End-to-End Learning Local Multi-view Descriptors for 3D Point Clouds. CVPR'2020 [paper] [code]
LRF-Net- Learning Local Reference Frames for 3D Local Shape Description and Matching. arxiv'2020 [paper]
DH3D: Deep Hierarchical 3D Descriptors for Robust Large-Scale 6DoF Relocalization. ECCV'2020 [paper] [code]
Distinctive 3D local deep descriptors. arxiv'2020 [paper] [code]
SpinNet: Learning a General Surface Descriptor for 3D Point Cloud Registration. CVPR'2021 [paper] [code]
PREDATOR: Registration of 3D Point Clouds with Low Overlap. CVPR'2021 [paper] [code]
Self-supervised Geometric Perception. CVPR'2021 [paper] [code]

Survey:

A Comprehensive Performance Evaluation of 3D Local Feature Descriptors. IJCV'2015 [paper]
Evaluating Local Geometric Feature Representations for 3D Rigid Data Matching. ICIP'2019 [paper]

Outlier Rejection

RANSAC: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. 1981 [paper]
Locally Optimized RANSAC. 2003 [paper]
Graph-cut RANSAC. CVPR'2018 [paper] [code]
MAGSAC: Marginalizing Sample Consensus. CVPR'2019 [paper] [code]
VFC: A Robust Method for Vector Field Learning with Application To Mismatch Removing. CVPR'2011 [paper]
In Search of Inliers: 3D Correspondence by Local and Global Voting. CVPR'2014 [paper]
FGR: Fast Global Registration. ECCV'2016 [paper] [code]
Ranking 3D Feature Correspondences Via Consistency Voting. PRL'2019 [paper]
An Accurate and Efficient Voting Scheme for a Maximally All-Inlier 3D Correspondence Set. TPAMI'2020 [paper]
GORE: Guaranteed Outlier Removal for Point Cloud Registration with Correspondences. TPAMI'2018 [paper] [code]
A Polynomial-time Solution for Robust Registration with Extreme Outlier Rates. RSS'2019 [paper]
Graduated Non-Convexity for Robust Spatial Perception: From Non-Minimal Solvers to Global Outlier Rejection. ICRA'2020 [paper]
TEASER: Fast and Certifiable Point Cloud Registration. T-RO'2020 [paper] [code]
One Ring to Rule Them All: Certifiably Robust Geometric Perception with Outliers. NeurIPS'2020 [paper]
SDRSAC: Semidefinite-Based Randomized Approach for Robust Point Cloud Registration without Correspondences. CVPR'2019 [paper] [code]
Robust Low-Overlap 3D Point Cloud Registration for Outlier Rejection. ICRA'2019 [paper]

Learning based (including 2D outlier rejection methods)

Learning to Find Good Correspondences. CVPR'2018 [paper] [code]
NM-Net: Mining Reliable Neighbors for Robust Feature Correspondences. CVPR'2019 [paper] [code]
OANet: Learning Two-View Correspondences and Geometry using Order-Aware Network. ICCV'2019 [paper] [code]
ACNe: Attentive Context Normalization for Robust Permutation-Equivariant Learning. CVPR'2020 [paper] [code]
SuperGlue: Learning Feature Matching with Graph Neural Networks. CVPR'2020 [paper] [code]
3DRegNet: A Deep Neural Network for 3D Point Registration. CVPR'2020 [paper] [code]
Deep Global Registration. CVPR'2020 [paper] [code]
3D Correspondence Grouping with Compatibility Features. arxiv'2020 [paper]
PointDSC: Robust Point Cloud Registration using Deep Spatial Consistency CVPR'2021 [paper] [code]

Survey

A Performance Evaluation of Correspondence Grouping Methods for 3D Rigid Data Matching. TPAMI'2019 [paper]

Graph Algorithms

A Graduated Assignment Algorithm for Graph Matching. TPAMI'1996 [paper]
A Spectral Technique for Correspondence Problems Using Pairwise Constraints. ICCV'2005 [paper] [code]
Balanced Graph Matching. NIPS'2006 [paper] [code]
Feature Correspondence via Graph Matching: Models and Global Optimization. ECCV'2008 [paper]
An Integer Projected Fixed Point Method for Graph Matching and MAP Inference. NIPS'2009 [paper] [code]
Optimal Correspondences From Pairwise Constraints. ICCV'2009 [paper]
Reweighted Random Walks for Graph Matching. ECCV'2010 [paper]
Maximal Cliques Based Rigid Body Motion Segmentation with a RGB-D Camera. ACCV'2012 [paper]
A Probabilistic Approach to Spectral Graph Matching. TPAMI'2013 [paper]
A Practical Maximum Clique Algorithm for Matching with Pairwise Constraints. arxiv'2019 [paper]
ROBIN: a Graph-Theoretic Approach to Reject Outliers in Robust Estimation using Invariants. arxiv'2020 [paper]
CLIPPER A Graph-Theoretic Framework for Robust Data Association. arxiv'2020 [paper]

End-to-End

Some papers perform end-to-end registration by directly predicting a rigid transformation aligning two point clouds without explicitly following the detection -- description -- outlier filtering pipepline. While they works well on object-centric datasets, the performance on real-world scene registration is not satisfactory.

PointNetLK: Robust & Efficient Point Cloud Registration using PointNet. CVPR'2019 [paper] [code]
Deep Closest Point: Learning Representations for Point Cloud Registration. ICCV'2019 [paper] [code]
PRNet: Self-Supervised Learning for Partial-to-Partial Registration. NeurIPS'2019 [paper] [code]
AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects. 3DV'2019 [paper] [code]
RPM-Net: Robust Point Matching using Learned Features. CVPR'2020 [paper] [code]
Feature-metric Registration: A Fast Semi-supervised Approach for Robust Point Cloud Registration without Correspondences. CVPR'2020 [code] [code]
Iterative Distance-Aware Similarity Matrix Convolution with Mutual-Supervised Point Elimination for Efficient Point Cloud Registration. ECCV'2020 [paper] [code]
Self-supervised Point Set Local Descriptors for Point Cloud Registration. arxiv'2020 [paper]
Learning 3D-3D Correspondences for One-shot Partial-to-partial Registration. arxiv'2020 [paper]
Robust Point Cloud Registration Framework Based on Deep Graph Matching. CVPR'2021 [paper] [code]

Randomized

RANSAC: Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography. 1981 [paper]
4PCS: 4-points Congruent Sets for Robust Pairwise Surface Registration. TOG'2008 [paper]
Model Globally, Match Locally: Efficient and Robust 3D Object Recognition. CVPR'2010 [paper] [code]
Super 4PCS: Fast Global Pointcloud Registration via Smart Indexing. CGF'2014 [paper] [code]

Probabilistic

Point Set Registration: Coherent Point Drift. TPAMI'2010 [paper] [code]
Robust Point Set Registration Using Gaussian Mixture Models. TPAMI'2011 [paper] [code]
A Generative Model for the Joint Registration of Multiple Point Sets. ECCV'2014 [paper]
Aligning the Dissimilar: A Probabilistic Method for Feature-based Point Set Registration. ICPR'2016 [paper]
A Probabilistic Framework for Color-Based Point Set Registration. CVPR'2016 [paper]
Density Adaptive Point Set Registration. CVPR'2018 [paper] [code]
HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration. ECCV'2018 [paper]
Robust Feature-Based Point Registration Using Directional Mixture Model. arxiv'2019 [paper]
FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization. CVPR'2019 [paper] [code]
PointGMM: a Neural GMM Network for Point Clouds. CVPR'2020 [paper] [code]
DeepGMR: Learning Latent Gaussian Mixture Models for Registration. ECCV'2020 [paper] [code]
Registration Loss Learning for Deep Probabilistic Point Set Registration. 3DV'2020 [paper] [code]
A Termination Criterion for Probabilistic PointClouds Registration. arxiv'2020 [paper]

Others

Gravitational Approach for Point Set Registration. CVPR'2016 [paper]
Accelerated Gravitational Point Set Alignment with Altered Physical Laws. ICCV'2019 [paper]
Fast Gravitational Approach for Rigid Point Set Registration with Ordinary Differential Equations. arxiv'2020 [paper]
Minimal Solvers for Mini-Loop Closures in 3D Multi-Scan Alignment. CVPR'2019 [paper]
Minimal Solvers for 3D Scan Alignment With Pairs of Intersecting Lines. CVPR'2020 [paper]
Learning multiview 3D point cloud registration. CVPR'2020 [paper] [code]
A Dynamical Perspective on Point Cloud Registration. arxiv'2020 [paper]
Plane Pair Matching for Efficient 3D View Registration. arxiv'2020 [paper]

Fine Registration

The fine registration methods (or local registration) produce highly precise registration results, given the initial pose between two point clouds.

Traditional

Point2Point ICP: A Method for Registration of 3-D Shapes. TPAMI'1992 [paper]
Point2Plane Object Modelling by Registration of Multiple Range Images. TPAMI'1992 [paper]
RPM: New Algorithms for 2D and 3D Point Matching: Pose Estimation and Correspondence. [paper]
Matching of 3-D Curves using Semi-differential Invariants. ICCV'1995 [paper]
The Trimmed Iterative Closest Point Algorithm. 2002 [paper]
Comparing ICP Variants on Real-World Data Sets. 2013 [paper]
Generalized-ICP. RSS'2009 [paper] [code]
Go-ICP: Solving 3D Registration Efficiently and Globally Optimally. ICCV'2013 [paper] [code]
AA-ICP: Iterative Closest point with Anderson Acceleration. ICRA'2018 [paper]
Point Clouds Registration with Probabilistic Data Association. IROS'2016 [paper] [code]
GH-ICP：Iterative Closest Point Algorithm with Global Optimal Matching and Hybrid Metric. 3DV'2018 [paper] [code]
NDT: The Normal Distributions Transform: A New Approach To Laser Scan Matching. IROS'2003 [paper]
Best Buddies Registration for Point Clouds. ACCV'2020 [paper]
Provably Approximated ICP. arxiv'2021 [paper]

Learning-based

DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration. ICCV'2019 [paper]

qsisi / awesome-point-cloud-registration