SIakovlev / udacity_deepRL_p1_navigation

Udacity Deep Reinforcment Learning Nanodegree: project 1

1 - Introduction

For this project, you will train an agent to navigate (and collect bananas!) in a large, square world.

Trained Agent

A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana. Thus, the goal of your agent is to collect as many yellow bananas as possible while avoiding blue bananas.

The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction. Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:

• 0 - move forward.
• 1 - move backward.
• 2 - turn left.
• 3 - turn right.

The task is episodic, and in order to solve the environment, your agent must get an average score of +13 over 100 consecutive episodes.

2 - Project Structure

• folder Banana_env contains simple banana environment with continious state space
• folder VisualBanana_env contains banana environment with pixel state representation (84*84 RGB image)
• folder data/models contains saived trained models
• folder reports contains saived training scores
• folder src contains all source code
  • replay_buffer.py contains 2 classes for experience replay: ReplayBuffer for regular experience replay, and PrioritizedReplayBuffer for prioritized experience replay
  • neural_net.py contains simple MLP and Convolution NNs
  • environment.py contains wrappers for 2 environments presented in this project
  • agent.py contains implementations of 4 algorithms: DQN, Double DQN, DQN+PER, Double DQN+PER
  • dqn.py contains common dqn routine used for training all the agents
  • train.py is a script for training any presented agent in any of 2 environments
  • play.py as a script for running trained agent

You can also:

• Import and save files from GitHub, Dropbox, Google Drive and One Drive
• Drag and drop markdown and HTML files into Dillinger
• Export documents as Markdown, HTML and PDF

Markdown is a lightweight markup language based on the formatting conventions that people naturally use in email. As John Gruber writes on the Markdown site

The overriding design goal for Markdown's formatting syntax is to make it as readable as possible. The idea is that a Markdown-formatted document should be publishable as-is, as plain text, without looking like it's been marked up with tags or formatting instructions.

This text you see here is actually written in Markdown! To get a feel for Markdown's syntax, type some text into the left window and watch the results in the right.

Tech

Dillinger uses a number of open source projects to work properly:

• AngularJS - HTML enhanced for web apps!
• Ace Editor - awesome web-based text editor
• markdown-it - Markdown parser done right. Fast and easy to extend.
• Twitter Bootstrap - great UI boilerplate for modern web apps
• node.js - evented I/O for the backend
• Express - fast node.js network app framework @tjholowaychuk
• Gulp - the streaming build system
• Breakdance - HTML to Markdown converter
• jQuery - duh

And of course Dillinger itself is open source with a public repository on GitHub.

Installation

Dillinger requires Node.js v4+ to run.

Install the dependencies and devDependencies and start the server.

$ cd dillinger
$ npm install -d
$ node app

For production environments...

$ npm install --production
$ NODE_ENV=production node app

Plugins

Dillinger is currently extended with the following plugins. Instructions on how to use them in your own application are linked below.

Plugin	README
Dropbox	plugins/dropbox/README.md
Github	plugins/github/README.md
Google Drive	plugins/googledrive/README.md
OneDrive	plugins/onedrive/README.md
Medium	plugins/medium/README.md
Google Analytics	plugins/googleanalytics/README.md

Development

Want to contribute? Great!

Dillinger uses Gulp + Webpack for fast developing. Make a change in your file and instantanously see your updates!

Open your favorite Terminal and run these commands.

First Tab:

$ node app

Second Tab:

$ gulp watch

(optional) Third:

$ karma test

Building for source

For production release:

$ gulp build --prod

Generating pre-built zip archives for distribution:

$ gulp build dist --prod

Docker

Dillinger is very easy to install and deploy in a Docker container.

By default, the Docker will expose port 8080, so change this within the Dockerfile if necessary. When ready, simply use the Dockerfile to build the image.

cd dillinger
docker build -t joemccann/dillinger:${package.json.version} .

This will create the dillinger image and pull in the necessary dependencies. Be sure to swap out ${package.json.version} with the actual version of Dillinger.

Once done, run the Docker image and map the port to whatever you wish on your host. In this example, we simply map port 8000 of the host to port 8080 of the Docker (or whatever port was exposed in the Dockerfile):

docker run -d -p 8000:8080 --restart="always" <youruser>/dillinger:${package.json.version}

Verify the deployment by navigating to your server address in your preferred browser.

127.0.0.1:8000

Kubernetes + Google Cloud

See KUBERNETES.md

Todos

• Write MORE Tests
• Add Night Mode

License

MIT

Free Software, Hell Yeah!