BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark
Nikita Cherniadev*, Nicholas Backshall*, Xiao Ma*, Yunfan Lu, Younggyo Seo, Stephen James
BiGym is a new benchmark and learning environment for mobile bi-manual demo-driven robotic manipulation. BiGym features 40 diverse tasks set in home environments, ranging from simple target reaching to complex kitchen cleaning. To capture the real-world performance accurately, we provide human-collected demonstrations for each task, reflecting the diverse modalities found in real-world robot trajectories. BiGym supports a variety of observations, including proprioceptive data and visual inputs such as RGB, and depth from 3 camera views.
For latest updates, check our project page: https://chernyadev.github.io/bigym/
pip install .
| Task | Description | Preview |
|---|---|---|
| ReachTarget | Reach the target with either left or right wrist. |  |
| ReachTargetSingle | Reach the target with specific wrist. |  |
| ReachTargetDual | Reach 2 targets, one with each arm. |  |
| StackBlocks | Move blocks across the table, and stack them in the target area. |  |
| MovePlate | Move the plate between two draining racks. |  |
| MoveTwoPlates | Move two plates simultaneously from one draining rack to the other. |  |
| FlipCup | Flip the cup, initially positioned upside down on the table, to an upright position. |  |
| FlipCutlery | Take the cutlery from the static holder, flip it, and place it back into the holder. |  |
| DishwasherOpen | Open the dishwasher door and pull out all trays. |  |
| DishwasherClose | Push back all trays and close the door of the dishwasher. |  |
| DishwasherOpenTrays | Pull out the dishwasher’s trays with the door initially open. |  |
| DishwasherCloseTrays | Push the dishwasher’s trays back with the door initially open. |  |
| DishwasherLoadPlates | Move plates from the rack to the lower tray of the dishwasher. |  |
| DishwasherLoadCups | Move cups from the table to the upper tray of the dishwasher. |  |
| DishwasherLoadCutlery | Move cutlery from the table holder to the dishwasher’s cutlery basket. |  |
| DishwasherUnloadPlates | Move plates from the tray of the dishwasher to a table rack. |  |
| DishwasherUnloadCups | Move cups from the upper tray of the dishwasher to the table. |  |
| DishwasherUnloadCutlery | Move cutlery from the cutlery basket to a tray on the table. |  |
| DishwasherUnloadPlatesLong | A full task of unloading a plate: picking up the plate from dishwasher, placing this plate into the rack located in the closed wall cabinet, and closing the dishwasher and cupboard. |  |
| DishwasherUnloadCupsLong | A full task of unloading a cup: picking up the cup, placing it inside the closed wall cabinet, and closing the dishwasher and cupboard. |  |
| DishwasherUnloadCutleryLong | A full task of unloading a cutlery: picking up a cutlery, placing it into the cutlery tray inside the closed drawer, and closing the dishwasher and drawer. |  |
| DrawerTopOpen | Open the top drawer of the kitchen cabinet. |  |
| DrawerTopClose | Close the top drawer of the kitchen cabinet. |  |
| DrawersAllOpen | Open all sliding drawers of the kitchen cabinet. |  |
| DrawersAllClose | Close all sliding drawers of the kitchen cabinet. |  |
| WallCupboardOpen | Open doors of the wall cabinet. |  |
| WallCupboardClose | Close doors of the wall cabinet. |  |
| CupboardsOpenAll | Open all drawers and doors of the kitchen set. |  |
| CupboardsCloseAll | Close all drawers and doors of the kitchen set. |  |
| PutCups | Pick up cups from the table and put them into the closed wall cabinet. |  |
| TakeCups | Take two cups out from the closed wall cabinet and put them on the table. |  |
| PickBox | Pick up a large box from the floor and place it on the counter. |  |
| StoreBox | Move a large box from the counter to the shelf in the cabinet below. |  |
| SaucepanToHob | Take the saucepan from the closed cabinet and place it on the hob. |  |
| StoreKitchenware | Take all items from the hob and place them in the cabinet below. |  |
| ToastSandwich | Use the spatula to put the sandwich on the frying pan. |  |
| FlipSandwich | Flip the sandwich in the frying pan using the spatula. |  |
| RemoveSandwich | Take the sandwich out of the frying pan. |  |
| GroceriesStoreLower | Place a random set of groceries in the cabinets below the counter. |  |
| GroceriesStoreUpper | Place a random set of groceries in cabinets and shelves on the wall. |  |
Directly instantiate the task of interest. Tasks are located in bigym/envs/.
from bigym.action_modes import TorqueActionMode
from bigym.envs.reach_target import ReachTarget
from bigym.utils.observation_config import ObservationConfig, CameraConfig
env = ReachTarget(
action_mode=TorqueActionMode(floating_base=True),
observation_config=ObservationConfig(
cameras=[
CameraConfig(
name="head",
rgb=True,
depth=False,
resolution=(128, 128),
)
],
),
render_mode=None,
)Use ActionModes to parameterise how you want to control your robot.
Please see simple example here: examples/replay_demo.py.
Demonstrations are automatically downloaded from GitHub releases.
When demos are requested by calling DemoStore.get_demos(). The current dataset will be cached locally at $HOME/.bigym/v0.0.0.
Demonstrations with custom observations and frequency are also cached to $HOME/.bigym/v0.0.0.
JointPositionActionMode(floating_base=True, absolute=True)JointPositionActionMode(floating_base=True, absolute=False)
Replay existing demos using GUI player.
python tools/demo_player/main.py
Record new demos in VR. Follow VR README to configure docker container to run this tool.
python tools/demo_recorder/main.py
On each PR, please ensure to bump the:
- Major version if you alter the existing interface in any way.
- Minor version if you have added new features (which didn't break the existing interface)
- Patch version for bug fixes.
Please ensure that you pass pre-commits before opening a PR: pre-commit run --all-files and that you pass all tests: pytest tests/ --run-slow.
- BiGym License (Apache 2.0) - This repository
- Mujoco Menagerie (Apache 2.0) - Models of robots and grippers
- 3D Assets Attributions (CC0, CC BY 4.0, CC BY NC 4.0) - 3D Assets
If you find our work helpful, please kindly cite us
@article{chernyadev2024bigym,
title={BiGym: A Demo-Driven Mobile Bi-Manual Manipulation Benchmark},
author={Chernyadev, Nikita and Backshall, Nicholas and Ma, Xiao and Lu, Yunfan and Seo, Younggyo and James, Stephen},
journal={arXiv preprint arXiv:2407.07788},
year={2024}
}