soheilesm / trcopo

This repository contains all code and experiments for Trust region Competitive policy optimization (TRCoPO) algorithm.

Trust Region Competitive Policy Optimization (TRCoPO)

This repository contains code of trust region competitve policy optimisation (TRCoPO) algorithm. The paper for competitive policy gradient can be found here, The code for Competitive Policy Gradient (CoPG) algorithm can be found here.

Experiment videos are available here

Dependencies

1. Code is tested on python 3.5.2.
2. Only Markov Soccer experiment requires OpenSpiel library, Other experiments can be run directly.
3. Require torch.utils.tensorboard

Repository structure

.
├── notebooks
│   ├── RockPaperScissors.ipynb
│   ├── MatchingPennies.ipynb
├── game                            # Each game have a saparate folder with this structure
│   ├── game.py                     
│   ├── copg_game.py                
│   ├── gda_game.py
│   ├── network.py
├── copg_optim
│   ├── copg.py 
│   ├── critic_functions.py 
│   ├── utils.py 
├── car_racing_simulator
└── ...

1. [Jupyter notebooks] are the best point to start. It contains demonstrations and results.
2. Folder [copg_optim] contains optimization code

How to start ?

Open jupyter notebook and run it to see results.

or

git clone "adress"
cd trcopo
cd RockPaperScissors
python3 trcopo_rps.py
cd ..
cd tensorboard
tensordboard --logdir .

You can check results in the tensorboard.

Experiment Demonstration

                            TRGDA vs TRGDA                                              TRCoPO vs TRCoPO

ORCA Car Racing

                               

Rock Paper Scissors

                              

Markov Soccer

                          

Matching Pennies