pytorch-AWAC
Advantage weighted Actor Critic for Offline RL implemented in pytorch.
A cleaner implementation for AWAC built on top of spinning_up SAC, and rlkit.
If you use this code in your research project please cite us as:
@software{Sikchi_pytorch-AWAC,
author = {Sikchi, Harshit and Wilcox, Albert},
doi = {10.5281/zenodo.5121023},
title = {{pytorch-AWAC}},
url = {https://github.com/hari-sikchi/AWAC}
}
Running the code
python run_agent.py --env <env_name> --seed <seed_no> --exp_name <experiment name> --algorithm 'AWAC'
Plotting
python plot.py <data_folder> --value <y_axis_coordinate>
The plotting script will plot all the subfolders inside the given folder. The value is the y-axis that is required. 'value' can be:
- AverageTestEpRet
Results
Environment: Ant-v2
Dataset: https://drive.google.com/file/d/1edcuicVv2d-PqH1aZUVbO5CeRq3lqK89/view
Plot [Compare to Figure 5 of the official paper]:
References
@article{SpinningUp2018, author = {Achiam, Joshua}, title = {{Spinning Up in Deep Reinforcement Learning}}, year = {2018} }
@article{nair2020accelerating, title={Accelerating Online Reinforcement Learning with Offline Datasets}, author={Nair, Ashvin and Dalal, Murtaza and Gupta, Abhishek and Levine, Sergey}, journal={arXiv preprint arXiv:2006.09359}, year={2020} }