TroddenSpade / RL-Microcontroller-DeepMind-Control

Deep RL with Microcontrollers for DeepMind Control Tasks

Requirements

• PyTorch
• Arduino
• Maixduino
• DeepMind Control
• Mojuco-Py

Development Boards

Arduino

• Forward Pass
• Backpropagation

Maixduino

• Forward Pass
• Backpropagation

Environments

• CartPole
• Acrobat
• Cheetah
• Hopper
• Walker
• Humanoid

Gym Wrapper

Implemented a Gym Wrapper for the DeepMind Control Suit environments. You can observe their status in the table below.

Mode	Task	Support
acrobot	swingup	✔️
acrobot	swingup_sparse	✔️
ball_in_cup	catch	✔️
cartpole	balance	✔️
cartpole	balance_sparse	✔️
cartpole	swingup	✔️
cartpole	swingup_sparse	✔️
cheetah	run	✔️
finger	spin	✔️
finger	turn_easy	✔️
finger	turn_hard	✔️
fish	upright	✔️
fish	swim	✔️
hopper	stand	✔️
hopper	hop	✔️
humanoid	stand	✔️
humanoid	walk	✔️
humanoid	run	✔️
manipulator	bring_ball	❌
pendulum	swingup	✔️
point_mass	easy	✔️
reacher	easy	✔️
reacher	hard	✔️
swimmer	swimmer6	✔️
swimmer	swimmer15	✔️
walker	stand	✔️
walker	walk	✔️
walker	run	✔️

Experiments

MuJoCo CartPole Test

demo.mp4

tinkercad.mp4

CartPole Environment

demo.mp4

Ball in Cup Environment

demo.mp4

Cheetah Environment

demo.mp4

References