We presented a system to generate the panorama pseudo-LiDAR point, with multiple monocular photos taken from different orientations serving as input. In this system, we successfully recovered and sparsified the point cloud from each direction and aligned them with the transformation matrix calculated from image features.
The project can be decomposed to serval parts.
Folder dataset contains the method to load data from KITTI, Waymo Open dataset and custom dataset.
Folder src contains the source python files for this project. In these files, we wrote API for liteflow, PointRCNN, Monodepth2 and Realsense camera.
The whole pipeline is shown in the figure.
python main.pypython bino_api.pypython open3d_utils.pySource images:
Point cloud recovered from setero images and comparison with ground truth LiDAR points.
Detection results
Source images:
Panorama point clouds recovered from five images with monodepth2 as depth prediction model.
Source images:
Fusion result:
ICP fusion results details:
