Cobaramin / Dota2Bot

Dota2 Bot

Dota2 Bot is an dota2 creep blocking AI bot was implement by reinforcement learning alogorithums via. Deep Deterministic Policy Gradients (DDPG). The environment was developed base on CreepBlockAI. It's part of my graduated project. (The Exploration Effect of Reinforcement Learning in Multiplayer Online Battle Arena Game).

Reffecence paper
Silver, David, Lever, Guy, Heess, Nicolas, Degris, Thomas, Wierstra, Daan, and Riedmiller, Martin. Deterministic policy gradient algorithms. In ICML, 2014.
Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. CoRR, abs/1509.02971, 2015.

Demo Video

Requirements

• Dota2
• Dota2 Workshop Tools DLC
• python3.6
• pip

Installation

Install libs

$ pip install -r requirement.txt

Setting (server/setting.py)

Correct settting is nessesery for corect running task such as

• training by uniform noise
• traing by Ornstein-Uhlenbeck
• testing (evaluating)

Paramter Setting

>>> BUFFER_SIZE = 100000
>>> GAMMA = 0.99  # Discounted Factor
>>> BATCH_SIZE = 200
>>> TAU = 0.001  # Target Network HyperParameters
>>> LRA = 0.0001  # Learning rate for Actor
>>> LRC = 0.001  # Lerning rate for Critic

Network config setting

>>> ACTION_DIM = 2  # x_pos , y_pos
>>> STATE_DIM = 11  # of sensors input
>>> ACTOR_HIDDEN1_UNITS = 150
>>> ACTOR_HIDDEN2_UNITS = 200
>>> CRITIC_HIDDEN1_UNITS = 150
>>> CRITIC_HIDDEN2_UNITS = 200

Lerning behavior setting

>>> TRAIN = 0
>>> EXPLORE = 20
>>> OU = 0
>>> MU = -10
>>> SIGMA = 30
>>> REPLACE_FREQ = 1
>>> BOOTSTRAP_FREQ = 5
>>> SAVE_FREQ = 1000

Run Flask server

$ python server/app.py

Load weights from pre-trained network

ex1. uniform noise with 20 noise scale with boostraping

$ python server/reload.py --timestamp 1521285961 --ep 151000

ex2. Ornstein-Uhlenbeck without boostraping

$ python server/reload.py --timestamp 1523632535 --ep 118000