CPP module
AI-student-wl opened this issue · comments
Hello, I don't understand the data reading part, so I want to directly experiment with the CPP module. My dataset is ScanNet. I wonder if you can tell me what the input parameters of the following function are. I only know that imageTensor is the input RGB image at present. I'm sorry to bother you like this. Thank you for your work in this field
CPP_training.py 116 ：extrinsic_channel = get_extrinsic_channel(imageTensor, focal_length, p_pt, extrinsic_para, self.CEILING_HEIGHT, augmentation=sample_dict['augmentation'])
The focal_length and p_pt are the camera intrinsic parameters. These two you could directly load from ScanNet camera files (forget the exact name of the file but each scene should have a file describes the camera parameters). In terms of the extrinsic_para, you need to read the pose file for each frame. The pose file should be specified in 4 x 4 matrix for ScanNet. In this implementation, we first convert the matrix into rotation in roll, pitch, yaw. So you could either do the same thing or directly load the matrix but make sure your camera coordinate system is consistent in your code. CEILING_HEIGHT is just a predefined parameters specifies the height of the room.
Hello, could you please help me to see the results? The left figure below is the pseudo depth map output by CPP, and the right figure is the depth map. I don't know whether this result is correct
This looks different from what I have in the paper but you might use a camera different coordinate system. Ideally, the horizon should have the largest value so the largest value should appear in the upper part of this image. But your encoded image has brighter area on the right. I think you might want to double check this. The easiest way to debug is to find the horizon, i.e., by giving different camera poses and check if the location of the horizon makes sense.