multi-commander / Multi-Commander

Multi & Single Agent Reinforcement Learning for Traffic Signal Control Problem

Multi-Commander

Traffic Signal Control based on Multi & Single Agent Reinforcement learning Algorithms

This is a project for Deecamp 2019 which cooperated with APEX Lib of Shanghai Jiaotong University and Tianrang Intelligence Company

Documentation for Q-Value based Method

DQN | DDQN | DuelingDQN | RayDQN

Documentation for PG based Method

PPO | DDPG | TD3 | SAC

Documentation for Distributed Method

IMPALA | A3C | APPO | Ape-X

Multi-Agent Method

QMIX | Gamma-Reward | PressLight

Docker image

this is a complete Docker image,you can start work quickly with it.
environment：ubuntu16.04 python3.6 Cityflow tensorflow1.14.0 Ray Gym
you need run your docker container keep port 8080,6006,8265 connected.

docker pull 957001934/road-commander

Demo video : bilibli

Installation

you can also built your environment,please follow these steps. This project is based on CityFlow which is a multi-agent reinforcement learning environment for large scale city traffic scenario, the algorithms are bases on Ray which is a fast and simple framework for building and running distributed applications.

CityFlow build from source

This guide is based on Ubuntu 16.04.

CityFlow has little dependencies, so building from source is not scary.

1. Check that you have python 3.6 installed. Other version of python might work, however, we only tested on python with version >= 3.6.
2. Install cpp dependencies

apt update && apt-get install -y build-essential libboost-all-dev cmake

3. Clone CityFlow project from github.

git clone --recursive https://github.com/cityflow-project/CityFlow.git

Notice that CityFlow uses pybind11 to integrate C++ code with python, the repo has pybind11 as a submodule, please use --recursive to clone all codes.

4. Go to CityFlow project’s root directory and run

pip install .

5. Wait for installation to complete and CityFlow should be successfully installed.

import cityflow
eng = cityflow.Engine

Ray Installation

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple -U ray  # also recommended: ray[debug]

Gym

git clone https://github.com/openai/gym
cd gym
pip install -e .