fccc0417 / mdnpg

Momentun-based Decentralized Natural Policy Gradient (MDNPG)

Code for paper "Decentralized Natural Policy Gradient with Variance Reduction for Collaborative Multi-Agent Reinforcement Learning"

Basic Requirements

• Python (3.7)
• Pytorch (1.4.0)
• Numpy (1.21.5)
• OpenAI Gym (0.18.0)

Code Structure

├─ CooperativeNavigation # Experiments on Cooperative navigation.
│ ├─ multiagent # Code for environment [REF].
│ │ ├─ core.py
│ │ ├─ environment.py
│ │ ├─ multi_discrete.py
│ │ ├─ policy.py
│ │ ├─ rendering.py
│ │ ├─ scenario.py
│ │ └─ scenarios
│ ├─ particle_envs # Code for environment [REF].
│ │ └─ make_particleworld.py
│ ├─ topology # Generate connectivity topology [REF].
│ │ ├─ connectivity
│ │ ├─ genconnectivity.py
│ │ └─ load_con.py
│ ├─ tools
│ │ └─ tool.py
│ ├─ nets.py # Policy network and value network.
│ ├─ rl_utils.py
│ ├─ momentum_npg.py # Class of MDNPG method.
│ ├─ mdnpg_main.py # Main file of MDNPG method.
│ ├─ momentum_pg.py # Class of MDPGT method [REF].
│ ├─ mdpgt_main.py # Main file of MDPGT method [REF].
│ ├─ rl_utils_vp.py
│ ├─ value_propagation_one_step.py # Class of one-step-version VP method.
│ ├─ value_propagation_multi_step.py # Class of multi-step-version VP method.
│ └─ value_propagation_main_on_policy.py # Main file of on-policy-version VP method.
├─ GridWorld # Experiments on GridWorld.
│ ├─ envs # Code for environment.
│ │ ├─ grid_maps
│ │ ├─ gridworld.py
│ │ └─ gridworld_4_test.py
│ ├─ multi_agents # Multi-agent GridWorld.
│ │ ├─ topology # Generate connectivity topology [REF].
│ │ ├─ nets.py # Policy network and value network.
│ │ ├─ rl_utils.py
│ │ ├─ momentum_npg.py # Class of MDNPG method.
│ │ ├─ mdnpg_main.py # Main file of MDNPG method.
│ │ ├─ momentum_pg.py # Class of MDPGT method [REF].
│ │ ├─ mdpgt_main.py # Main file of MDPGT method [REF].
│ │ ├─ pg_entropy.py # Class of PG with entropy method.
│ │ └─ pg_entropy_main.py # Main file of PG with entropy method.
│ ├─ single_agent # Single-agent GridWorld.
│ │ ├─ momentum_npg_discrete.py # Momentum-based NPG.
│ │ ├─ momentum_pg_discrete.py # Momentum-based PG.
│ │ ├─ pg_entropy_discrete.py # PG with entropy.
│ │ ├─ ppo_discrete.py # PPO.
│ │ └─ srvrnpg_discrete.py # SRVR-NPG.
│ └─ tools
│ └─ tool.py
├─ MountainCar # Experiments on MountainCarContinuous (single agent).
│ ├─ momentum_npg_continuous.py # Momentum-based NPG.
│ ├─ momentum_pg_continuous.py # Momentum-based PG.
│ ├─ ppo_continuous.py # PPO.
│ └─ srvrnpg_continuous.py # SRVR-NPG.
└─ README.md