A modern & clean implementation of the PILCO Algorithm in TensorFlow.
Unlike PILCO's original implementation which was written as a self-contained package of MATLAB, this repository aims to provide a clean implementation by heavy use of modern machine learning libraries.
In particular, we use TensorFlow to avoid the need for hardcoded gradients and scale to GPU architectures. Moreover, we use GPflow for Gaussian Process Regression.
The core functionality is tested against the original MATLAB implementation.
- Install venv
virtualenv -p python3 venv
source venv/bin/activate- Install requirements
pip install -r requirements.txt
python setup.py develop- You might also need to install openai gym
pip install gym- You might also need to install mujoco click here
- In the case that mujoco and mujoco-py fail to build on macos, change the MacOS SDK to 10.14.sdk
Before using, or installing, PILCO, you need to have Tensorflow 1.13.1 installed (either the gpu or the cpu version). It is recommended to install everything in a fresh conda environment with python>=3.7. Given Tensorflow, PILCO can be installed as follows
git clone https://github.com/nrontsis/PILCO && cd PILCO
python setup.py develop
The examples included in this repo use OpenAI gym 0.15.3 and mujoco-py 2.0.2.7. Once these dependencies are installed, you can run one of the examples as follows
python examples/inverted_pendulum.py
While running an example, Tensorflow might print a lot of warnings, some of which are deprecated. If necessary, you can suppress them by running
tf.logging.set_verbosity(tf.logging.ERROR)right after including TensorFlow in Python.
The following people have been involved in the development of this package:
See the following publications for a description of the algorithm: 1, 2, 3