madan96 / bisimulation-transfer

Implementation of the paper Castro et al. "Using bisimulation for policy transfer in MDPs." AAAI-2010

Bisimulation Policy Transfer

Python implementation of the paper Castro et al. Using bisimulation for policy transfer in MDPs AAAI-2010
Note: This repository is still under development.

Getting Started

This project requires Python 3.5
Follow the instructions on installing the necessary prerequisites and running the code.

Requirements

For PyEMD, please clone my PyEMD fork and follow its installation instructions.

To install the other necessary prerequisites use pip install -r requirements.txt

How to Run

Train Q-Learning agent on small world using

python q_learning.py --env-name FourSmallRooms_11

Run Policy Transfer for 8 to 44 states using

python run_transfer.py --transfer optimistic --tgt-env FourLargeRooms --solver pyemd

Results

Policy transfer result for transfer between FourSmallRoom env to FourLargeRoom env

Policy transfer result for transfer between FourSmallRoom env to ThreeLargeRoom env

References

• Using Bisimulation for Policy Transfer in MDPs [Paper]
• Custom RL-Gridworld repo [Link]

Credits

Rishabh Madan
Anirban Santara
Pabitra Mitra
Balaraman Ravindran