marco-milanesi / lqr-ppo

Git repository for LQR and Reinforcement Learning labs. Code for modeling human movement and solving optimization using PPO algorithm.

Linear Quadratic Regulator (LQR) and Reinforcement Learning

This repository contains two laboratory exercises focused on understanding optimal control theory, specifically the Linear Quadratic Regulator (LQR) and integrating the Proximal Policy Optimization (PPO) reinforcement learning algorithm

Team

Table of Contents

1. Linear Quadratic Regulator (LQR)
2. Reinforcement Learning through PPO
3. Requirements
4. License
5. Contact

Linear Quadratic Regulator (LQR)

./Source/Forearm Movements LQR Model.ipynb introduces LQR by first performing basic manipulations of the data and then manually applying the LQR. I The data used in this laboratory exercise come from a 2011 experiment where participants aimed towards a thin line using 5 different strategies ranging from fast to precise. The data can be found in the Dataset folder.

The lab has 4 parts: the first part is about basic manipulations of the data, and visualizing that data. The second and third part are ”courses”, and we will treat them together. The fourth part is about applying what you just learned in the course.

(back to top)

Reinforcement Learning through PPO

./Source/Reinforcement Learning through PPO.ipynb focuses on using the model-free reinforcement learning algorithm, PPO, to solve the optimization problem. The laboratory exercise uses the stable-baselines3 library, which contains an implementation of PPO.

(back to top)

---

The results of the two laboratory exercises are represented and commented on within the notebook.

Requirements

Both laboratory exercises were developed in Google Colab and the required installations are detailed within each notebook.

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Contact

• Marco Milanesi - [e-mail]
• Andrea Campanelli - [e-mail]

(back to top)