A repository for the reproduction of async RL algorithms. Each application under the directory /applications/ is built for solving a spefic domain/task. And scripts under /examples/ give cases for training.
These days, I want to investigate the use of FeuDal, a typical hierarchical RL algorithm for goal-conditioned learning. However, there is no official implementation for this work, and many third-party implementations totally cannot work for solving Atari games. I list them below to help users to get rid of wasting their time reading this garbage:
- lweitkamp/feudalnets-pytorch
- davidhershey/feudal_networks
- dnddnjs/feudal-montezuma
- vtalpaert/pytorch-feudal-network
After a deeper review of their code, I found the reason is that there is a misleading of the pytorch-implementation of A3C, an asynchronous RL algorithm as the basis of FeuDal. And I also list them as follows (totally cannot work for solving Gym, too!):
NOTE: never implement A3C with a shared optimizer like the above! Please refer to the official torch guides of IMPLEMENTING BATCH RPC PROCESSING USING ASYNCHRONOUS EXECUTIONS
In this repo, I give a functional and lightweight implementation of A3C, which works for solving gym, at least!
| Status | Algorithm | Example |
|---|---|---|
| DONE | A3C | PYTHONPATH=.:asyncrl python examples/a3c_gym.py --num-processes 2 |
| 🕓 | IMPALA | ... |
| 🕓 | APPO (Sample-Factory implementation) | ... |
Solving CartPole-v1 via A3C (num_worker=2)
