Goals
This in an implementation of an OpenAI gym environment to train controllers for Roboy. They provide standardized interface definitions for RL: https://github.com/openai/gym/tree/master/gym/envs
The goals of this repo are:
- To provide flexibility for you to train controllers for different robots (e.g. MSJ Platfrom, Upper Body, etc.).
- Second, it is straightforward to parallelize the environments s.t. you can scale the controller training with more compute resources.
- Lastly, we kept it easy to implement communication to your simulation engine of choice (e.g. to MuJoCo, Gazebo, etc.).
Installation
Python3.5 is required. Either use python3
and pip3
variants,
or activate a Python virtual environment. If you are using the docker container deepandreinforced/rl:latest
, you
do not need to install this repo again. Otherwise:
python3 -m pip install -r requirements.txt
python3 -m pip install -e .
Installing this repo as a pip package is necessary if you would like to use the environment constructor form:
env = gym.make('msj-control-v1')
Structure
- The class
RoboyEnv
implements gym'sgym.GoalEnv
. It is the central class in this repo.
simulations/
Directory SimulationClient
is a client interface to talk to our simulations. It provides the flexibility to change simulation engines by subclassing it.RosSimulationClient
is a subclass ofSimulationClient
that uses ROS services to step and reset the simulation. We have used so far CARDSflow as our simulation engine (not part of this repo).
robots/
Directory RoboyRobot
is the interface for any robot. You can add new robots by subclassing from it.MsjRobot
is a subclass ofRoboyRobot
with concrete dimensions and boundaries.
Class Diagram
The RoboyEnv
uses a SimulationClient
to read values from the simulation.
The SimulationClient
depends upon a RoboyRobot
to parse the numbers it
receives from the simulation into a meaningful RobotState
Run tests
Before running the tests, make sure you have sourced ROS2 with the ROS messages
of Roboy. In the docker container deepandreinforced/rl:latest
, this can be done
with the command source_ROS2_ROBOY_WS
.
To run the unit tests:
pytest
The repo also has integration tests. To run them, you need a CARDSflow simulation running as well as the bridge between ROS1 and ROS2. Then, to run the tests, do:
cd gym_roboy/envs/tests/
bash run_all_tests.sh