[Feature Request] Can the gym environment provide depth images and point clouds?

[StarCycle]

RLBench/rlbench/gym/rlbench_env.py

Lines 50 to 63 in 46cb9d9

 elif observation_mode == 'vision': 
     self.observation_space = spaces.Dict({ 
         "state": spaces.Box( 
             low=-np.inf, high=np.inf, 
             shape=obs.get_low_dim_data().shape), 
         "left_shoulder_rgb": spaces.Box( 
             low=0, high=1, shape=obs.left_shoulder_rgb.shape), 
         "right_shoulder_rgb": spaces.Box( 
             low=0, high=1, shape=obs.right_shoulder_rgb.shape), 
         "wrist_rgb": spaces.Box( 
             low=0, high=1, shape=obs.wrist_rgb.shape), 
         "front_rgb": spaces.Box( 
             low=0, high=1, shape=obs.front_rgb.shape), 
         }) 

Although the original RLBench environment includes depth images and point clouds in the observation, the gym version does not provide such information. They are important if researchers want to render images from new viewpoints (e.g., Robotic View Transformer). Could you please add them in the code?

I can modify the code and pull a request if you like it.

Kind regards,
Zhuoheng