x35f / unstable_baselines

Re-implementations of SOTA RL algorithms.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool


License

Unstable Baselines(USB) is designed to serve as a quick-start guide for Reinforcement Learning beginners and a codebase for agile algorithm development. The algorithms strictly follows the original implementations, and the performance of Unstable Baselines matches those in the original implementations. USB is currently maintained by researchers from lamda-rl.


Features

  1. Novice-friendly: USB is written in simple python codes. The RL training procedures are highly decoupled, waiting to be your first RL playground.
  2. Stick to the original implementations: USB is as a benchmark framework for RL, thus the re-implementations strictly follows the original implementations. Tricks to achieve a higher performance are not implemented.
  3. Customized Environments: You can customized you own environment as long as it has Gym-like interfaces.

Implementation Details

Baseline RL Continuous Action Space Discrete Action Space Image Input Status
DQN Stable
VPG Stable
DDPG Stable
TD3 Stable
TRPO Stable
PPO Stable
SAC Stable
REDQ Stable
Option Critic - - - Developing
Model Based RL Continuous Action Space Discrete Action Space Image Input Status
MBPO Updating
Meta RL Continuous Action Space Discrete Action Space Image Input Status
PEARL Updating
MAML - - - Developing

*Updating: the algorithm is being developed to adapt to the latest USB version, and will be "Stable" soon

*Developing: the algorithm is being implemented, and will appear on the project soon


Supported environment benchmarks


Performance

MuJoCo

Quick Start

Install

git clone --recurse-submodules https://github.com/x35f/unstable_baselines.git
cd unstable_baselines
conda env create -f env.yaml 
conda activate usb
pip install -e .

To run an algorithm

In the directory of the algorithm

python3 /path/to/algorithm/main.py /path/to/algorithm/configs/some-config.py args(optional)

For example

cd unstable_baselines/baselines/sac
python3 main.py configs/Ant-v3.py --gpu 0

or for the ease of aggregating logs

python3 unstable_baselines/baselines/sac/main.py unstable_baselines/baselines/sac/configs/Ant-v3.py --gpu 0

Install environments (optional)

#install metaworld for meta_rl benchmark
cd envs/metaworld
pip install -e .

TODO List

  • Add Documentation

About

Re-implementations of SOTA RL algorithms.


Languages

Language:Python 99.2%Language:Ruby 0.8%Language:Shell 0.0%