Question about the up direction of "front" and "back" camera in renderer.py

[Chan]

Hi @imankgoyal, thanks for your great work.

While reviewing the code in rvt/mvt/renderer.py, I noticed in the get_cube_R_T function that the up direction for the five cameras is defined as follows:

    for view in elev_azim:
        if view in ["left", "right"]:
            up.append((0, 0, 1))
        else:
            up.append()

It appears that the front and back cameras are aligned along the Y-axis as defined in RLBench. However, I'm curious about the up direction being set to (0, 1, 0) for these cameras. Since this also represents the Y-axis, wouldn't this configuration lead to a potential conflict in the camera orientation?

I apologize for any inconvenience this question may cause, and I appreciate your time in addressing this query.
Thank you!

[Ankit Goyal]

Hi,

Thanks for your interest in our work.

Overall, the up direction for all cameras (except the top one) is Y-axis. The particular code block for left and right is for compensating for the rotation and making sure that the input parameter for look_at_view_transform is correct. I guess the best way to make sure everything is correct would be to visualize the virtual images.

Best,
Ankit

[Chan]

Thanks a lot! I will check about it.