7enTropy7 / BipedalWalker

This is how you teach a dumb bot to walk on two feet.

BipedalWalker

This is how I taught a dumb bot to walk on two feet.

Bipedal Walker is an OpenAI Gym environment where an agent learns to control a bipedal 2D character to reach the end of an obstacle course. What makes this challenging is that the agent only receives limbs coordinates along with the Lidar information. The agent has to learn to balance,walk,run,jump on its own without any human intervention.

Environment Specifications

Reward is given for moving forward, total 300+ points up to the far end. If the robot falls, it gets -100. Applying motor torque costs a small amount of points, more optimal agent will get better score. State consists of hull angle speed, angular velocity, horizontal speed, vertical speed, position of joints and joints angular speed, legs contact with ground, and 10 lidar rangefinder measurements. There's no coordinates in the state vector.

Algorithms

• Deep Q Learning
• NeuroEvolution of Augmenting Topologies
• Deep Deterministic Policy Gradients
• Augmented Random Search

Cloning

$ git clone https://github.com/7enTropy7/BipedalWalker.git

Directory Contents

$ cd BipedalWalker/
$ tree
.
├── ARS
│   ├── main.py
│   ├── normalizer.py
│   ├── policy.py
│   └── Runs
│       ├── openaigym.video.0.31413.video032000.meta.json
│       └── openaigym.video.0.31413.video032000.mp4
├── DDPG
│   ├── actor_checkpoint.pth
│   ├── critic_checkpoint.pth
│   ├── __pycache__
│   │   └── walker.cpython-37.pyc
│   ├── test.py
│   ├── train.py
│   └── walker.py
├── DQN
│   ├── ai.py
│   ├── Bipedal-dqn-testing.h5
│   └── main.py
├── NEAT
│   ├── config-bipedal-walker.txt
│   └── neat_walker.py
├── README.md
└── requirements.txt

6 directories, 18 files

Installing Dependencies

$ pip3 install -r requirements.txt

Testing Instructions

$ cd BipedalWalker/ARS/
$ python3 main.py

Outputs

Augmented Random Search

Trained for around 4 hours. This is Perfection!!

Deep Deterministic Policy Gradients

After training for around 2000 episodes (6 hours of training on my low-end CPU).

---

Author

• Unnikrishnan Menon

License

This project is licensed under the MIT License - see the LICENSE file for details