Installation Dependencies can be installed with the following command: conda env create -f conda_env.yml Usage The code supportd two benchmark: gym and atari, please checkout the corresponding directory for each domain's usage.