Repository for reactive task and motion planning making use of active inference for symbolic planning, and a new multi-modal MPPI for motion planning. Rollouts are evaluated in IsaacGym, a parallelizable physics simulator.
First, clone the repo in your folder and create the conda environment.
cd <project_folder>
git clone https://github.com/tud-amr/m3p2i-aip.git
conda create -n m3p2i-aip python=3.8
conda activate m3p2i-aip
This project requires the source code of IsaacGym. Check for the prerequisites and troubleshooting. Download it from https://developer.nvidia.com/isaac-gym, unzip and paste it in the thirdparty
folder. Move to IsaacGym and install the package.
cd <project_folder>/m3p2i-aip/thirdparty/IsaacGym_Preview_4_Package/isaacgym/python
pip install -e.
Then install the current package by:
cd <project_folder>/m3p2i-aip
pip install -e.
Now you are ready to test an example file, where you can drive the robot around with ASDW keys.
cd <project_folder>/m3p2i-aip/examples
python3 example_key.py
If you want to test the TAMP framework, you will need two instances of Isaac Gym, one for throwing the rollouts and deriving the optimal solution, and one for updating the "real system". Please run the commands below in two terminals from the scripts
folder with activated python environment.
Run this terminal first:
cd <project_folder>/m3p2i-aip/scripts
conda activate m3p2i-aip
python3 reactive_tamp.py --robot $(robot_type) --task $(task_type)
Then run the second terminal:
cd <project_folder>/m3p2i-aip/scripts
conda activate m3p2i-aip
python3 sim.py --robot $(robot_type) --task $(task_type)
Specifically, you can test the following:
python3 reactive_tamp.py --robot point --task push
python3 sim.py --robot point --task push
Note
Pushing will always fail if the initial position is in the corner.
python3 reactive_tamp.py --robot point --task pull
python3 sim.py --robot point --task pull
Note
Pulling will always fail if the goal is in the corner.
python3 reactive_tamp.py --robot point --task hybrid --multimodal True
python3 sim.py --robot point --task hybrid --multimodal True
You can play with the cube using ASDW keys.
python3 reactive_tamp.py --robot panda --task reactive_pick
python3 sim.py --robot panda --task reactive_pick
You can play with the cube using ASDW keys.
python3 reactive_tamp.py --robot panda --task reactive_pick --multimodal True
python3 sim.py --robot panda --task reactive_pick --multimodal True
If you find the code useful, please cite:
@article{zhang2024multi,
title={Multi-Modal MPPI and Active Inference for Reactive Task and Motion Planning},
author={Zhang, Yuezhe and Pezzato, Corrado and Trevisan, Elia and Salmi, Chadi and Corbato, Carlos Hern{\'a}ndez and Alonso-Mora, Javier},
journal={IEEE Robotics and Automation Letters},
year={2024},
publisher={IEEE}
}
- Isaac-mppi: an MPPI implementation that uses IsaacGym as a dynamic model (paper, website, code).
- Biased-mppi: an MPPI implementation whose sampling distribution is informed with ancillary controllers (paper, website, code).
- AIP: an Active Inference planner for decision making (paper, video, code).
We thank the pioneers (Grady Williams, pytorch_mppi, storm) who have paved the way and the future newcomers who will propel MPPI forward!