notmahi / disk

PyTorch implementation for "Discovery of Incremental Skills" (DISk) algorithm from ICLR 2022 paper "One After Another: Learning Incremental Skills for a Changing World"

Discovering Incremental Skills (DISk) implementation in PyTorch

This is PyTorch implementation of Discovering Incremental Skills (DISk) [OpenReview] [Presentation] [Website] [Arxiv].

If you use this code in your research project please cite us as:

@inproceedings{shafiullah2021one,
  title={One After Another: Learning Incremental Skills for a Changing World},
  author={Shafiullah, Nur Muhammad Mahi and Pinto, Lerrel},
  booktitle={International Conference on Learning Representations},
  year={2021}
}

Requirements

We assume you have access to a gpu that can run CUDA 11.3. Then, the simplest way to install all required dependencies is to create an anaconda environment and activate it:

conda env create -f conda_env.yml
conda activate disk

Unfortunately, our experiments require you to install MuJoCo version 1.50 because of a bug in OpenAI Gym MuJoCo environments. You can download the older MuJoCo version and use a free license key, all from here.

Instructions

To train an SAC agent on the Ant-v3 task run:

python train.py env=ant

This will produce exp folder, where all the outputs are going to be stored including train/eval logs, and evaluation episode videos. Some data will also be stores in Weights and Biases for further analysis.

Results

A video is worth a thousand figures, so here you go.

Ant

Swimmer

Cheetah