Unity ML-Agents: Curriculum Training Project

Overview

This project demonstrates the implementation of curriculum training using Unity's ML-Agents toolkit. The focus is on progressively training an AI agent to navigate complex environments, enhance its walking abilities, and utilize ray perception for obstacle avoidance and checkpoint navigation. The project leverages the Proximal Policy Optimization (PPO) algorithm to optimize agent performance through various training phases.

Features

Phase 1: Basic Navigation - The agent learns to navigate to a target location in a simple environment.
Phase 2: Balanced Walking - Introduction of walking mechanics; the agent must maintain balance and proper posture while navigating.
Phase 3 (Incomplete): Complex Navigation - The agent uses ray perception to navigate through an environment with dynamic obstacles and checkpoints.

Installation

Clone this repository to your local machine using:

git clone https://github.com/yourusername/unity-ml-agents-curriculum-training.git

Prerequisites

Unity 2018.4 and later
Python 3.9.13 (virtual environment)
ML-Agents

Setup

For training and inference:

After creating the first new Unity Project, replace the folders under our repository.
Create a virtual environment using Python 3.9.13. If you have an existing python version, you may create virtual environment with specified python version here.
- ```
python -m venv "venv"
venv/scripts/activate
```
- Double check your environment with the correct version
- ```
python --version
```
Now that the python version is correctly installed, we may get these following pip requirements
- ```
pip3 install mlagents
pip3 install torch torchvision torchaudio
```
- (Optional) You may choose to use GPU for inference as well, please check Pytorch CUDA for your configuration.
```
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
```
- ```
pip3 install protobuf==3.20.3
```
- Once all above is installed, you should be able to get a prompt with the following:
```
mlagents-learn -h
```
- ```
pip3 install onnx
```

For inference only:

After creating the first new Unity Project, replace the folders under our repository.
Choose one of the brains under our Assets folder and drag it into the agent's inspector.

Enter play mode to check with the brain model

Results

Curriculum Training - Phase 1: Basic Navigation

curriculum_01_pressed.mp4

Curriculum Training - Phase 2: Balanced Walking

curriculum_02_pressed.mp4

Curriculum Training - Phase 3: Complex Navigation with Obstacles

curriculum_03_pressed.mp4

Folder Description

Brians contains trained models corresponding to each curriculum.
Materials contains color settings for objects.
Prefabs contains all objects we created, e.g. robots, enviornment.
Scripts contains all the code we write.
- AgentController is the main file to train the agent robot.
- CheckpointController, Goal Controller are files to make updates on status of checkpoint/goal button, so that we can add points when the robot hits the wall
- WallController is a file to detect the collision so that we can deduct points when the robot hits the wall
- LeftFootController,RightFootController are files to deduct if agent is standing so that we can add or deduct points for walking.
- Camera Controller is code to control the camera to move along with agent.

MissTiny / RL_robot