Inference on custom data

[Adit Doshi]

Hello,

Thank you for sharing your amazing work. I am just trying to inference on custom RGB data. But I checked in the example that the model is using ['keyframe', 'keyframe_pose', 'keyframe_intrinsics', 'frames', 'poses', 'intrinsics', 'sequence', 'image_id'] for inference. I am just curious to know is it possible to run the inference pipeline just for RGB data?
Thank you

[Nan Yang]

Hi, thanks for your interest in our work!

To answer your question more precisely, may I ask what do you mean by "RGB data" exactly? Do you mean images without poses and intrisincs?

[Adit Doshi]

Question 1: Simply RGB data (no poses no intrinsics)

Is it possible to run the inference on such data?

Question 2: I want to collect real-world data for inference can you please tell me what all data I need to collect along with RGB frames? Does it need lidar data also? If possible can you brief what you mean by intrinsic, keyframe_pose, and poses?

[Felix Wimbauer]

Q1: It is possible to run inference on such data as long as you know the intrinsics for the camera (Tutorial by OpenCV). However, you will have to run an odometry system to obtain the pose information (camera rotation and position). We recommend to use the open-source implementation of DSO (we tested this) to compute the poses (this will give the same format for the poses that is used in the implementation of the KITTI Odometry dataset).

Q2: Neither for training, nor for inference you need lidar data. We only use it to evaluate the predicted depth maps.