mac137 / lidar_camera_calibration-1

Camera LiDAR Calibration using ROS, OpenCV, and PCL

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

ROS Camera LIDAR Calibration Package

Setup

Run the following to clone the lidar_camera_calibration package in ros_workspace/src directory.

cd ~/ros_workspace/src
git clone https://github.com/heethesh/lidar_camera_calibration

cd ~/ros_workspace/
catkin_make
source devel/setup.bash

Make sure you have the ROS bag file in lidar_camera_calibration/bagfiles folder. Then you can use the following launch files. This package assumes that the bag file has atleast the following topic names and message types by default, these can be modified in the launch scripts.

/sensors/camera/camera_info (sensor_msgs/CameraInfo)
/sensors/camera/image_color (sensor_msgs/Image)
/sensors/velodyne_points    (sensor_msgs/PointCloud2)

Play ROS Bag File

This launch file will only play the rosbag record file.

roslaunch lidar_camera_calibration play_rosbag.launch bagfile:=/path/to/file.bag

Run Camera Calibration

This launch file will play the rosbag record and runs the camera_calibration package from ROS.

roslaunch lidar_camera_calibration camera_calibration.launch

23 images were automatically selected by the calibrator and the results are stored in lidar_camera_calibration/calibration_data/camera_calibration folder. The following results were obtained:

Camera Matrix

484.130454    0.000000  457.177461
  0.000000  484.452449  364.861413
  0.000000    0.000000    1.000000

Distortion Coefficients

-0.199619  0.068964  0.003371  0.000296  0.000000

Update the ROS Bag File

This script will update the camera matrices and the distortion coefficients in the /sensors/camera/camera_info topic and creates a new bag file in the same location.

rosrun lidar_camera_calibration update_camera_info.py <original_file.bag> <calibration_file.yaml>

Display Camera Calibration

This launch file will play the updated rosbag record, run image_proc for camera image rectification and displays the rectified and unrectified images.

roslaunch lidar_camera_calibration display_camera_calibration.launch

YouTube Link for Camera Calibration Demo

Calibrate Camera-LiDAR Point Correspondences

This script will perform calibration using the matplotlib GUI to pick correspondences in the camera and the LiDAR frames. You first need to play the rosbag record in another terminal.

roslaunch lidar_camera_calibration play_rosbag.launch bagfile:=/path/to/file.bag
rosrun lidar_camera_calibration calibrate_camera_lidar.py --calibrate

Press [ENTER] to launch the GUIs and pick the corresponding points by selecting the four corner points of the checkerboard in both the camera and the LiDAR frames. 16 corresponding points were selected for calibration at varying position and depths of the checkerboard. One such set of points picked are shown below. OpenCV's PnP RANSAC + refinement using LM is used to find the rotation and translation transforms between the camera and the LiDAR. Since OpenCV's function rectifies the images internally, the 2D points are picked from the unrectified image. Additional, the rectify flag can be set to True while creating the GUI process to pick points from a rectified image.

NOTE: The point files are appended and the extrinsics estimates are calculated and refined continuously using a RANSAC approach.

YouTube Link for Camera-LiDAR Calibration GUI Demo

The point correspondences are saved as following:

  • Image Points: lidar_camera_calibration/calibration_data/lidar_camera_calibration/img_corners.npy
  • LiDAR Points: lidar_camera_calibration/calibration_data/lidar_camera_calibration/pcl_corners.npy

The calibrated extrinsics are saved as following:

  • lidar_camera_calibration/calibration_data/lidar_camera_calibration/extrinsics.npz
    • 'euler' : Euler Angles (RPY rad)
    • 'R' : Rotation Matrix
    • 'T' : Translation Offsets (XYZ m)

The following calibrated extrinsics were obtained:

Rotation Matrix

-9.16347982e-02  -9.95792677e-01  -8.74577923e-05
 1.88123595e-01  -1.72252569e-02  -9.81994299e-01
 9.77861226e-01  -9.00013023e-02   1.88910532e-01

Euler Angles (RPY rad)

-0.44460865  -1.35998386   2.0240699

Translation Offsets (XYZ m)

-0.14614803  -0.49683771  -0.27546327

Display Camera-LiDAR Projection

This launch file will play the updated rosbag record, run calibrate_camera_lidar.py in projection mode and displays the LiDAR point cloud projected on to the image. A static transform is set up between the world and the velodyne frame which needs to be updates with the values above in the format X Y Z Y P R within the launch file.

roslaunch lidar_camera_calibration display_camera_lidar_calibration.launch

YouTube Link for Camera-LiDAR Projection Demo

TODO

  • Shift to Rviz point-picker for point cloud. Matplotlib GUI picker is not very convinient to use.

About

Camera LiDAR Calibration using ROS, OpenCV, and PCL


Languages

Language:Python 73.3%Language:CMake 26.7%