Project 1 - Udacity Deep Reinforcement Learning Nanodegree

Collecting Bananas with Deep Q Networks!

Problem Formulation

The agent is moving on a 2D surface, surrounded by walls from each side. The agent must collect YELLOW bananas while avoiding DARK bananas. Each time the agent collects a YELLOW banana, it will receive +1, and receives -1 collecting DARK banana, and it receives 0 otherwise.

State Space

A 37 dimensional vectors, contains Velocity of the Agent and sensory information about objects in the vicinity of the Agent.

Sample observation looks like this :

 1.         0.         0.         0.         0.84408134 0.
 0.         1.         0.         0.0748472  0.         1.
 0.         0.         0.25755    1.         0.         0.
 0.         0.74177343 0.         1.         0.         0.
 0.25854847 0.         0.         1.         0.         0.09355672
 0.         1.         0.         0.         0.31969345 0.
 0.

Action Space

Agent can be controlled with 4 action inputs:

0 - move forward.
1 - move backward.
2 - turn left.
3 - turn right.

Solve Criteria

The Environment is considered solved if the average score for 100 consecutive episode reaches 13. We also consider the task is strongly solved, if an agent scores more than 17 in 100 consecutive episodes (its average score).

Instructions and Structure

Notebooks

Use Navigation-Train.ipynb notebook for Training.
Use Navigation-Test.ipynb notebook for testing trained agents.
For Report check out Report.ipynb notebook.
Benchmark.ipynb notebook contains code for performing Benchmarks.

Scripts file

agents.py contains a code for a Generic Q Agent.
brains.py contains the definition of Neural Networks (Brains) used inside an Agent.

Folders

trained_agents folder contains saved weights for trained agents.
images folder contains images used in the notebooks.
movies folder contains recorded movies from each Agent.

Seting up Environment

It is highly recommended to create a separate python environment for running codes in this repository. The instructions are the same as in the Udacity's Deep Reinforcement Learning Nanodegree Repository. Here are the instructions:

Create (and activate) a new environment with Python 3.6.

Linux or Mac:

conda create --name drlnd python=3.6
source activate drlnd

Windows:

conda create --name drlnd python=3.6 
activate drlnd

Follow the instructions in this repository to perform a minimal install of OpenAI gym.

Here are quick commands to install a minimal gym, If you ran into an issue, head to the original repository for latest installation instruction:
```
 pip install box2d-py 
 pip install gym
```

Clone the repository (if you haven't already!), and navigate to the python/ folder. Then, install several dependencies.

git clone https://github.com/taesiri/udacity_drlnd_project1
cd drlnd-project-1/
pip install .

Create an IPython kernel for the drlnd environment.

python -m ipykernel install --user --name drlnd --display-name "drlnd"

Open the notebook you like to explore.

Video

See it in action here:

taesiri / udacity_drlnd_project1