faster-than-real-time discussion

[Paweł Gajewski]

I'd like to continue here the topic of faster-than-real-time simulation started as a side topic in #50.

1. Are there any concerns that running a simulation faster than real time will introduce some weird or inaccurate behaviors?
2. From my experience the qiBullet with Pepper spawned doesn't let me run much faster. Do you know what could cause it to be slow or how to speed it up? UPDATE: I just benchmarked it (GUI vs no-GUI) and it runs ~7x faster without rendering/GUI, so I guess that is "much faster".
3. In DIRECT mode there is a sleep between consecutive steps. It is 1./10000. sec. Why?

Background to my questions:
The purpose of running simulation fast is to train Reinforcement Learning algorithms. Because they require a lot of environment interactions doing RL in real-time is intractable. Most, if not all, simuators used in RL allow for running simulations much faster than real-time during the training phase for that reason.

[Maxime Busy]

1. Depending on how you speed up the simulation, it won't necessarily introduce weird/inaccurate behaviours, but definitely can. If you manually step the simulation, but don't change the Bullet timestep, my guess is that you won't have any problems. If you do change the timestep however (see the setTimeStep and setPhysicsEngineParameter methods in the PyBullet doc), depending on the content of your simulated instance and the physics engine parameters you might.

2. Main difference between the GUI and DIRECT modes: in GUI mode an extra thread is spawned by Bullet (only for Windows and Linux, not OSX), handling the rendering (and automatically stepping the simulation if setRealTimeSimulation is called with True). You don't have any rendering in DIRECT mode, no extra thread is spawned, meaning that you have to manually step by default. You may have very different results for your benchmark depending on your OS (my guess is that this kind of difference will be even greater on MacOS, where no extra thread is spawned for the rendering). What OS did you use ?

3. I assume that you're referring to _stepSimulation, running in a thread (let's call it t2, and our main thread t1) when DIRECT mode is used with auto-stepping: this is a dirty solution (stepping manually from the main thread is better). The two threads are running in the same process. To run, t2 waits for t1 to release Python's GIL (Global Interpreter Lock), and in turn t1 waits for t2 to release the GIL. The sleep has been added to t2 to release the GIL. qiBullet has been modified since then, I don't think that the sleep is still required.

I fully understand the constraints of RL algorithms training, and yes many simulators allow to run simulations faster than real time. But there is always a trade-off between your simulation speed and its "accuracy". You can run Bullet faster than real time, but again, depending on your hardware, the simulated environment, the physics parameters, the simulation time step, etc. you might have something faster but physically inconsistent.

A common approach in RL to speed up your training is to use multiple environments running in parallel to train your agent (suppressing the need to run "faster than real-time simulations"). This approach might actually be more suited for what you're trying to achieve.

The gitter is a better place to discuss about this, issues shouldn't be used as discussion topics. Closing this issue, let's continue to chat there