Introduction
This repository implements Lee et al. Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning. In ICLR, 2020 in TensorFlow==1.12.0.
@inproceedings{lee2020network,
title={Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning},
author={Lee, Kimin and Lee, Kibok and and Shin, Jinwoo and Lee, Honglak},
booktitle={ICLR},
year={2020}
}
Preliminaries
This code is based on CoinRun platform.
After installing all packages in CoinRun,
replace coinrun.cpp, coinrunenv.py, config.py, policies.py, ppo2.py, random_ppo2.py, and train_random.py with files in sources/ in this repository.
Training PPO agents
Vanilar PPO
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num-levels 500
Vanilar PPO + CutOut
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num-levels 500 -uda 1
Vanilar PPO + Dropout
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num-levels 500 -dropout 0.1
Vanilar PPO + BN
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num-levels 500 -norm 1
Vanilar PPO + L2
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num_levels 500 -l2 0.0001
Vanilar PPO + Grayout
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num-levels 500 -ubw 1
Vanilar PPO + Inversion
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num_levels 500 -ui 1
Vanilar PPO + Color Jitter
python -m coinrun.train_agent --run-id myrun --save-interval 1 --num-levels 500 -uct 1
PPO + ours
python -m coinrun.train_random --run-id myrun --save-interval 1 --num_levels 500 -lstm 2
Test on unseen environments
Please note that the train_flag is only set during evaluation to indicate that the evaluation is to be done using a different set of themes than the ones used for training.
python -m coinrun.enjoy --test-eval --restore-id myrun -num-eval N -rep K -train_flag 1