Predicting Object Interactions with Behavior Primitives: An Application in Stowing Tasks

Haonan Chen, Yilong Niu, Kaiwen Hong, Shuijing Liu, Yixuan Wang, Yunzhu Li, Katherine Driggs-Campbell

CoRL 2023, Oral | Website | Paper

Demo

Real-World Demo

Our Method

Prerequisites

Operating System: Ubuntu 20.04.
Python: 3.8.
Environment: robosuite-1.2.

Installation Steps

Create the conda environment:

conda env create -f environment.yml
conda activate stow

Update .bashrc:

echo 'export PYTHONPATH="${PYTHONPATH}:/path/to/stowing"' >> ~/.bashrc

Usage

Dataset Preparation for GNN

Sample and Build Dataset:

bash perception/scripts/run_sample.sh [push|sweep|insert]

Manual Inspection (Optional):
- Consolidate visualizations:
```
perception/scripts/inspect_perception.sh
```
- Review videos and record problematic indices in dump/perception/inspect/inspect.txt.

Construct GNN Dataset:

python perception/scripts/auto_patch_make.py

Training the Dynamics Model

You can find our pre-collected dataset here.

Execute the desired script by specifying the task (push, sweep, or insert):

python dynamics/scripts/hyperparameter_<task>.py

Code Structure

config/: Configuration files for perception and dynamics.
dynamics/: Scripts to train and evaluate the GNN.
perception/: The perception module of Stowing.
utils/: Utilities and visualization functions.

Citation

If you find our code or paper useful, please cite:

@inproceedings{
    chen2023predicting,
    title={Predicting Object Interactions with Behavior Primitives: An Application in Stowing Tasks},
    author={Haonan Chen and Yilong Niu and Kaiwen Hong and Shuijing Liu and Yixuan Wang and Yunzhu Li and Katherine Rose Driggs-Campbell},
    booktitle={7th Annual Conference on Robot Learning},
    year={2023},
    url={https://openreview.net/forum?id=VH6WIPF4Sj}       
}

Acknowledgement

The code is adapted from RoboCraft.

haonan16 / Stow