Create nuScenes-inspired bird's-eye view (BEV) images using the Carla Simulator.
(Left: Carla_nuScenes_BEV, Right: no_rendering_mode provided by Carla/PythonAPI)
The nuScenes dataset has become a staple in the autonomous vehicle and robotics community for developing and benchmarking algorithms. However, generating similar rasterized bird's-eye view images in a simulation environment like Carla is not straight-forward.
The Carla_nuScenes_BEV repository aims to bridge this gap by providing a simple, efficient, and customizable way to generate nuScenes-style bird's-eye view images in the Carla Simulator.
-
The core functionality is contained in the
BirdeyeRender
class. You can easily integrate it into your codebase with the providedBEVGenerator
module. Here's a quick example to get you started:bev_gen = BEVGenerator(hero_actor) image = bev_gen.render() # Visualize with matplotlib plt.imshow(image) plt.show()
Note: the first run on each map could take some time, as we need to parse the OpenDrive file to gain a full-view of the map.
We will cache the rendered result in png format, so that on the next run, we could simply load the layer.
-
For those interested in generating a continuous stream of BEV frames during a running episode in Carla, refer to the example script
test.py
. -
If you have specific needs, feel free to extend or wrap the
BirdeyeRender
class for greater flexibility.
-
Currently, the rendering process relies solely on the Pygame library. Exploring alternative rendering engines for improved performance is on the roadmap.
-
Investigate the feasibility and benefits of switching to OpenCV for image manipulation and rendering.
-
Implement a more efficient caching mechanism to speed up frame rendering.