This repository contains code allowing you to train, test, and visualize OpenAI Gym environments (games) using the NEAT algorithm and its variants.
The two goals of this project are
-
Make this work as simple as possible. For example, the name of the NEAT configuration file defaults to the name of the environment.
-
Make the code run fast, by simultaneously evaluating the fitnesses of the population on multiprocessor machines.
To get started you should install neat-python and PUREPLES from source. Then do the following:
% python3 neat-evolve.py
This will run neat-python on the default CartPole-v1 environment using the parallel fitness evaluator, so you can take advantage of all the cores on your computer.
Once evolution finishes, you can try out your evolved network by doing:
% python3 neat-test.py models/CartPole-v1<fitness>.dat
where <fitness> is the fitness of your evolved network.
NEAT-Gym supports HyperNEAT via the --hyper option and
and ES-HyperNEAT via the
--eshyper option.
NEAT-Gym supports
Novelty Search
via the --novelty option. To use this option, your Gym environment should provide a
step_novelty method. Like the ordinary step method, step_novelty should
accept an action as input; however, instead of returning a tuple state, reward, done, info
it should return state, reward, behavior, done, info, where behavior is the behavior
of the agent at the end of the episode (for example, its final position in the
maze), or None before the end of the episode.
gym-nsmaze provides this capability, using the medium/hard maze environments in the Novelty Search paper.
The neat_gym module exports two functions:
-
read_file loads a .dat file from command-line arguments, returning the evolved network and the name of the envrionment used to evovle it (as well as flags for recording a move and suppressing the display).
-
eval_net takes the output of
read-fileand runs an episode of the environment using the network.
To make these functions available other projects (e.g., for 3D animation of your environment during testing), do:
% sudo python3 setup.py install
You will also need to create a NEAT configuration file for your environment. As usual, the easiest way to do this is to take something that works (like the config file for CartPole-v1) and modify it to do what you want.