Questions about scene layout and rendering

[Aryan Jain]

1. When I run a task, the default scene appears to have a yellow background and a gray workstation. Is it possible to change the urdf files to get the same visuals shown in this repository's README?
2. Is it possible to get each image frame (that goes into making the video demo) and action (taken internally by the sim) from an oracle-executed path?

Thank you so much! I've been loving Vima so far!

[Yunfan Jiang]

Hey there, thank you for your interest in our project! Regarding your questions

1. Yes. But please note that we use NVISII to enhance the visual appearance in README for better demonstration. The data generation and training are still performed on the original rendering.

2. Yes. Regarding per-frame image, if you just want to record videos, you can simply pass record_gui = True to vima_bench.make(). However, if you want per-frame images for other purposes such as training, please note that it will significantly slow down the simulation because we need to render RGB images every frame. See here for example code.

Regarding per-frame action, because we use action primitives (pick-place and sweep), see here for how pick-place works under the hood, here for how sweep works. In general, both primitives call an internal function to move robot joints.

Let me know if you have further questions!