BLOG POST DOCUMENTATION : Consider reading our detailed blog post about this project.
Amazon SageMaker for Battlesnake AI
This project shows how to build and deploy an AI for the platform Battlesnake on AWS with Amazon Sagemaker!
It is ready to deploy and contains learning materials for AI enthusiasts.
What is Battlesnake? (taken from battlesnake.com):
Battlesnake is an autonomous survival game where your snake competes with others to find and eat food without being eliminated. To accomplish this, you will have to teach your snake to navigate the serpentine paths created by walls, other snakes, and their own growing tail without running out of energy.
Intention
This project contains a ready-to-use AI for Battlesnake as well as a development environment that can be used to modify and improve the AI. The included AI makes movement decisions in two steps:
- Run a Neural Network Model
- Run Heuristics (ie: additional code that can override the Neural Network's predicted best action so that your snake avoids colliding with walls, eats food if it is safe to do so, attempts to eat a competitor snake, ...)
Several pre-trained neural network models are provided within this project as well as some default heuristics. These pre-trained models (snakes) are not designed to win the Battlesnake competition, so you'll have to improve them in order to have a chance of winning. The training environment is provided for you to make modifications and to retrain the neural network models in order to obtain better results.
Project Organisation
This project can be used in three steps:
- Step 1 - Deploy a Snake : Deploy a snake with pre-trained AI. After this step, you'll be able to participate in a Battlesnake game! Don't skip this step! This step also deploys a local environment which is needed for the following steps.)
- Step 2 - Build some Heuristics : Write some rules for your snake to navigate common scenarios, visualize your snake, and publish the upgraded version of your snake!
- Step 3 - Upgrade your Reinforcement Learning Model : This is the most rewarding step. Retrain your snake with different settings, visualize your snake, and publish an upgraded version!
Reinforcement learning packages
The repository includes two training method and framework options:
- the in-built SageMaker reinforcement learning package (based on RLlib and Tensorflow)
- an implementation written in MXNet
You can choose which one to use when you deploy the stack.
Architecture
If you use Steps 1-3, you will have the following deployed within your AWS account:
Cost
This project has been designed to run within the AWS free tiers for some time. Refer to each step above in order to understand the costs that may be incurred after the free tiers are exceeded.
Content
The included Jupyter notebooks are bundled with a step-by-step Battlesnake visualizer:
With the Battlesnake visualizer, you can load any initial state and test how your snake will behave in various situations (See the Visualize your algorithm section for the full list of features.)
The source code of the project is organized as follows:
CloudFormation # contains the templates and scripts to automate deployment
BattlesnakeGym # OpenAI Gym environment that simulates the Battlesnake game
Heuristics # your custom heuristics
LambdaGateway # code that exposes the Snake API
MXNet # training environment with the MXNet implementation
> TrainingEnvironment # contains the Jupyter notebook for model training
> InferenceEndpoint # contains the code for the Snake Endpoint
> HeuristicDevelopment # contains the Jupyter notebook for heuristics development
RLlib
> TrainingEnvironment # contains the Jupyter notebook for model training
> InferenceEndpoint # contains the code for the Snake Endpoint
> HeuristicDevelopment # contains the Jupyter notebook for heuristics development
License
This project is licensed under the Apache-2.0 License.