Scannet 3D box projection visualization

[zianggu]

Hi there,
Thanks for the wonderful work!!
I just want to know whether I could get the ground truth projection of the 3d box in the image space.
I have gone through the data preprocessing and I found that we have the rgb image at each frame and the intrinsic and extrinsic parameters, as well as the 3d coordinates of the bounding box in the 3d space.
As a result, I firstly get 8 corners of the 3d box in the 3d space and then get the projected 8 concerns in the 2d space

def get_3d_bounding_box_vertices(bbox,index):

# Given a 3D bounding box, compute the 8 vertices

    x_center, y_center, z_center, x_size, y_size, z_size ,label = bbox[index]

    x_half = x_size / 2

    y_half = y_size / 2

    z_half = z_size / 2

    vertices = np.array(

    [
    (x_center + x_half, y_center + y_half, z_center + z_half),
    (x_center + x_half, y_center + y_half, z_center - z_half),
    (x_center + x_half, y_center - y_half, z_center + z_half),
    (x_center + x_half, y_center - y_half, z_center - z_half),
    (x_center - x_half, y_center + y_half, z_center + z_half),
    (x_center - x_half, y_center + y_half, z_center - z_half),
    (x_center - x_half, y_center - y_half, z_center + z_half),
    (x_center - x_half, y_center - y_half, z_center - z_half),
    ])
    return vertices

def project_3d_point_to_image(point_3d, intrinsic_matrix, extrinsic_matrix):

    # Project a 3D point to the image plane using intrinsic and extrinsic matrices

    point_3d_homogeneous = np.append(point_3d, 1)  # Convert to homogeneous coordinates

    projected_point_homogeneous = np.dot(intrinsic_matrix[:3,:], np.dot(np.linalg.inv(extrinsic_matrix), point_3d_homogeneous))

    projected_point = projected_point_homogeneous[:2] / projected_point_homogeneous[2]

    return list(projected_point)

But the problem is that the bounding box is not aligned with the image.....
I just want to know how you guys do the visualizations, could you provide me some code to visualize the image?
Thank you so much for your time and contribution to the community!!!
Sincerely,
ZiAng

[Danila Rukhovich]

Hi @ZiAng1-G ,
You can follow our visulization function here.

[zianggu]

@filaPro Thank you so much for your reply, I have gone through all the files that you mentioned. I tried my best to work with them but it is really difficult for me to simply use them. Sorry for my lack of knowledge.
I just want to know if it is possible to get the box just from 'posed_images' folder and the 'scannet_instance_data' folder created at the intermediate step of the data preprocessing.
Thank you again for your time and response!!!! This really helped me a lot!! '
Sincerely,
ZiAng

[Danila Rukhovich]

Please follow our intrinsic and extrinsic definitions here, then thier multiplication here and finally their multiplication with box corners here.

[zianggu]

Thanks for the reply