Software and tasks for dexterous multi-fingered hand manipulation, powered by MuJoCo.
dexterity builds on dm_control and provides a collection of modular components that can be used to define rich Reinforcement Learning environments for dexterous manipulation. It also comes with a set of standardized tasks that can serve as a performance benchmark for the research community.
An introductory tutorial is available as a Colab notebook:
The recommended way to install this package is via PyPI:
pip install dexterityWe provide a Miniconda environment with Python 3.8 for development. To create it and install dependencies, run the following steps:
git clone https://github.com/kevinzakka/dexterity
cd dexterity
conda env create -f environment.yml # Creates a dexterity env.
conda activate dexterity
pip install .The MuJoCo dexterity suite is composed of the following core components:
models: MuJoCo models for dexterous hands and PyMJCF classes for dynamically customizing them.inverse_kinematics: Inverse kinematics library for multi-fingered hands.effectors: Interfaces for controlling hands and defining action spaces.
These components, in conjunction with dm_control, allow you to define and customize rich environments for reinforcement learning. We facilitate this process by providing the following:
task: Wrappers overcomposer.Taskthat simplify the creation of generic dexterous tasks as well as goal-reaching based tasks (e.g., successive object reorientation).manipulation: A library of pre-defined, benchmark RL environments geared towards dexterous manipulation. For an overview of the available tasks, see the task library.
Our hope is to grow the benchmark over time with crowd-sourced contributions from the research community -- PR contributions are welcome!
A large part of the design and implementation of dexterity is inspired by the MoMa library in dm_robotics.