This is the starter kit for the NeurIPS 2020 - Procgen competition hosted on AIcrowd
16 simple-to-use procedurally-generated gym environments which provide a direct measure of how quickly a reinforcement learning agent learns generalizable skills. The environments run at high speed (thousands of steps per second) on a single core.
These environments are associated with the paper Leveraging Procedural Generation to Benchmark Reinforcement Learning (citation). Compared to Gym Retro, these environments are:
- Faster: Gym Retro environments are already fast, but Procgen environments can run >4x faster.
- Non-deterministic: Gym Retro environments are always the same, so you can memorize a sequence of actions that will get the highest reward. Procgen environments are randomized so this is not possible.
- Customizable: If you install from source, you can perform experiments where you change the environments, or build your own environments. The environment-specific code for each environment is often less than 300 lines. This is almost impossible with Gym Retro.
git clone https://github.com/AIcrowd/neurips2020-procgen-starter-kit.git
./utils/setup.sh
- Clone this repository
git clone https://github.com/AIcrowd/neurips2020-procgen-starter-kit.git
-
Download and install Miniconda (if you don't have it already)
-
Create a new environment with python 3.7
conda create -n procgen -y
conda activate procgen
conda install python=3.7 -y
- Install dependencies
conda activate procgen
pip install ray[rllib]==0.8.6
pip install procgen
- Install the DL framework of you choice (tensorflow, pytorch, etc.,)
./run.sh --train
./run.sh --rollout
Please refer to the instructions here for advanced users.
To add a custom model, create a file inside models/
directory and name it models/my_vision_network.py
.
Please refer here for a working implementation of how to add a custom model. You can then set the custom_model
field in the experiment yaml to my_vision_network
to cause that model to be used.
Please refer to the instructions here
This is a very thin wrapper over RLLib. Hence, you have all the RLLib config options available in the experiment configuration (YAML) file to play with. Please refer to this example for an explanation on the structure and available options in the experiment config file.
.
βββ aicrowd_helpers # Helper files (DO NOT EDIT)
βββ algorithms # Directory to implement your custom algorithm/trainable/agent
β βββ custom_random_agent
β βββ random_policy
β βββ __init__.py
β βββ registry.py # Register your custom agents here
βββ envs
β βββ __init__.py
β βββ framestack.py # Example for using custom env wrappers
β βββ procgen_env_wrapper.py # Base env used during evaluations (DO NOT EDIT)
βββ experiments # Directory contaning the config for different experiments
β βββ impala-baseline.yaml # Baseline using impala
β βββ procgen-starter-example.yaml # Sample experiment config file
β βββ random-policy.yaml # Sample random policy config file
βββ models # Directory to implement custom models
β βββ impala_cnn_tf.py
β βββ impala_cnn_torch.py
β βββ my_vision_network.py
βββ preprocessors # Directory to implement your custom observation wrappers
β βββ __init__.py # Register your preprocessors here
β βββ custom_preprocessor.py
βββ utils # Helper scripts for the competition
β βββ setup.sh # Setup local procgen environment using `conda`
β βββ submit.sh # Submit your solution
β βββ teardown.sh # Remove the existing local procgen environment using `conda`
β βββ validate_config.py # Validate the experiment YAML file
β βββ loader.py
βββ Dockerfile # Docker config for your submission environment
βββ aicrowd.json # Submission config file (required)
βββ callbacks.py # Custom Callbacks & Custom Metrics
βββ requirements.txt # These python packages will be installed using `pip`
βββ rollout.py # Rollout script (DO NOT EDIT)
βββ run.sh # Entrypoint to your submission
βββ train.py # Script to trigger the training using `rllib` (DO NOT EDIT)
Your repository should have an aicrowd.json file with following fields:
{
"challenge_id" : "evaluations-api-neurips-2020-procgen",
"grader_id": "evaluations-api-neurips-2020-procgen",
"authors" : ["aicrowd-bot"],
"description" : "NeurIPS 2020: Procgen Competition Agent",
"docker_build" : false
}
This file is used to identify your submission as a part of the NeurIPS 2020 Procgen Competition Challenge. You must use the challenge_id
, and grader_id
as specified above.
By default we will run your code in an Ubuntu 18.04 environment with tensorflow==1.14.0
, ray[rllib]==0.8.5
and procgen
installed.
If you want to run your submissions in a customized environment, first head to aicrowd.json
and set docker_build
to true
. This flag tells that you need a custom environment.
You can specify your software environment by using Dockerfile``requirements.txt
. Available options are
requirements.txt
: We will usepip install -r requiremens.txt
to install your packages.Dockerfile
: We will build the docker image using the specified Dockerfile. If you have a Dockerfile in your repository, any other automatic installation will not be triggered. This means that you will have to include installation steps for packages inrequirements.txt
yourself.
A sample Dockerfile
and a corresponding requirements.txt
are provided in this repository for you reference.
The evaluator will read the value of EXPERIMENT
from run.sh
in the repository. During the training phase, we will use the file provided by EXPERIMENT
variable to launch the training. During the rollouts phase, we will use the most recent checkpoint and start rollouts with that.
- Setup an AIcrowd GitLab account if you don't have one by going to https://gitlab.aicrowd.com/
- Add your SSH key
- If you accept the challenge rules, click the
Participate
button on the AIcrowd contest page
# Optional: Start working with starter kit as starting point.
git clone git@github.com:AIcrowd/neurips2020-procgen-starter-kit.git
cd neurips2020-procgen-starter-kit
# Add AICrowd git remote endpoint
git remote add aicrowd git@gitlab.aicrowd.com:<your-aicrowd-username>/neurips-2020-procgen-starter-kit.git
git push aicrowd master
./utils/submit.sh "<your submission message>"
For example,
./utils/submit.sh "impala-ppo-v0.1"
Create a submission by pushing a tag to your repository with a prefix submission-
. An example is shown below (you can keep repository name as you desire or have multiple repositories):
# Create a tag for your submission and push
git tag -am "submission-v0.1" submission-v0.1
git push aicrowd master
git push aicrowd submission-v0.1
# Note : If the contents of your repository (latest commit hash) does not change,
# then pushing a new tag will not trigger a new evaluation.
- You should now be able to see the details of your submission in the repository's issues page.
https://gitlab.aicrowd.com/<your-aicrowd-username>/neurips-2020-procgen-starter-kit/issues
and something along the lines of :
Happy Submitting!! π