amanks-20 / SURGE_2022

🗂 SURGE'22 Research Project under Prof. Salil Goel, Dept. of Civil Engineering, IIT Kanpur

SURGE-22 RESEARCH PROJECT

• Application No: 2230061

• Name: Aman Kumar Singh

• Mentor: Prof. Salil Goel

• Department: Civil Engineering

Research Proposal for SURGE 2022

Title: Adding Colour Information to LiDAR using Camera Calibration

Aim: Add colour information to 3D LiDAR models using camera calibration and machine learning tools.

Research Problem: The 3D models generated using LiDAR lack colour and texture information in them. The research problem is to add colour information to the models using LiDAR-Camera Calibration.

Abstract: In recent years, 3D sensing system has aroused increasing attention due to their vast potential applications, such as autonomous driving and mobile robotics. These tasks have high demands for various applications in different field domains. Nowadays, with the popularity of crew-less vehicles, the navigation problems inherent in mobile robots are gathering even greater attention. One of the fundamental problems is the localisation or calibration between different sensors. LiDAR technology can gather 3D points with an effective range of up to 200 meters. In addition, LiDAR can be used in low-textured scenes and scenes with varying lighting conditions. However, the 3D model data generated by LiDAR is sparse and lacks colour information. A camera is a portable and cheap device that can obtain colour information. However, it needs to correspond to feature points during calculation, which will be time- consuming and sensitive to light. A combination of cameras and LiDAR requires obtaining transformation parameters between the coordinate systems of the two kinds of sensors. The calibration procedure leads to the determination of the transformation parameters, namely the rotation matrix and translation vector, the alignment of the two coordinate systems, and the correspondence between 3D points and 2D images. The 3D point cloud of the LiDAR is combined with the 2D image of the camera to create a 3D LiDAR model with colour information.

Keywords: LiDAR-Camera; Calibration.

References:

1. https://www.mdpi.com/1424-8220/20/21/6319
2. https://arxiv.org/abs/2101.04431
3. https://ieeexplore.ieee.org/document/8665256
4. https://github.com/heethesh/lidar_camera_calibration
5. https://in.mathworks.com/help/vision/ug/camera-calibration.html
6. https://docs.opencv.org/4.x/dc/dbb/tutorial_py_calibration.html