arya-ebrahimi / Triplesumo

Multi-agent physical-contact cooperation in adversarial environment

TripleSumo:

a virtual multi-agent RL platform based on OpenAI/Gym and Mujoco ¹

This game is introduced in publication Learning Cooperative Behaviours in Adversarial Multi-agent Systems (full text). This game aims to establish a virtual environment for intestivating multi-agent cooperation in physical contact-rich adversarial environment, with reinforcement learning interface ported to OpenAI/Gym. In this game, two week players are supposed to team up and play against a strong player in sumo game.

Demo of results:

The result after training the green agent for 3000 epochs;

The result after training both the green and red agents for fighting;

The result after training the blue agent to join the ongoing game.

You're welcome to visit the author's Youtube page to find more about her work. Contact her at [e-mail] if you have inquiry.

Steps of installing triplesumo:

1. Download Mujoco200, rename the package into mujoco200, then extract it in /home/your_username/.mujoco/ , then download the license into the same directory
2. Add export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/your_username/.mujoco/mujoco200/bin to your ~/.bashrc, and then source ~/.bashrc
3. Use Anaconda to create a virtual environment 'triple_sumo' with conda env create -f triplesumo2021.yml; Then conda activate triple_sumo.
4. git clone https://github.com/niart/triplesumo.git and cd triplesumo
5. Use the envs foler of this repository to replace the gym/envs installed in your conda environment triplesumo.
6. To train blue agent in an ongoing game between red and green, run cd train_bug, thenpython runmain2.py.
7. If you meet error Creating window glfw ... ERROR: GLEW initalization error: Missing GL version, you may add export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGLEW.so to ~/.bashrc, then source ~/.bashrc.

key algorithm: The reward function is in gym/envs/mojuco/triant.py; The training algorithm is in train_bug/DDPG4.py.

If you want to cite this game:

@misc{triplesumo,
  howpublished = {Wang, N., Das, G.P., Millard, A.G. (2022). Learning Cooperative Behaviours in Adversarial Multi-agent Systems. In: Pacheco-Gutierrez, S., Cryer, A., Caliskanelli, I., Tugal, H., Skilton, R. (eds) Towards Autonomous Robotic Systems. TAROS 2022. Lecture Notes in Computer Science(), vol 13546. Springer, Cham. https://doi.org/10.1007/978-3-031-15908-4_15} 

An overview of TripleSumo interface:

Rewards along training the newly added player with DDPG:

Wining rate of the team(red+blue) during training and testing:

Steps the team needed to win along training the newly added player:

Footnotes

1. This project is an extension of platform Robosumo with new interfaces. ↩