wurina-github/FGTrans

Functional Grasp Transfer across a Category of Objects from Only One Labeled Instance

Rina Wu, Tianqiang Zhu, Wanli Peng, Jinglue Hang, Yi Sun † ；
Dalian University of Technology
† corresponding author
IEEE Xplore

Our initial research has been accepted by [IEEE Robotics and Automation Letters(RAL)](https://ieeexplore.ieee.org/document/10079105).

Overview
Requirements
Data Preprocess
Training--Learning Categorical Dense Correspondence
Post processing of generated data
Acknowledgment
Citation
License

Overview

To assist or replace human beings in completing various tasks, research on the functional grasp synthesis of dexterous hands with high degree-of-freedom (DoF) is necessary and challenging. The dexterous functional grasp requires not only that the grasp is stable but more importantly facilitates the functional manipulation after grasping. Such work still relies on manual annotation when collecting data. To this end, we propose a category-level multi-fingered functional grasp transfer framework, in which we only need to label the hand-object contact relationship on functional parts of one object, and then transfer the contact information through the dense correspondence of functional parts between objects, so as to achieve the functional grasp synthesis for new objects based on the transferred hand-object contact information. We verify this method on three categories of representative objects through simulation experiments and achieve successful functional grasps by labeling only one instance in each category.

Requirements

Ubuntu 18.04 or 20.04
Python 3.7
PyTorch 1.8.1
PyTorch3D
torchmeta 1.8.0
NVIDIA driver version >= 460.32
CUDA 10.2

Data Preprocess

Download source meshes and grasp labels for knife, bottle categories from ShapeNetCore dataset or Baidu:-ecgf.
Please download the trained model of this project from Baidu:-ecgf. And arrange the files as follows:

|-- FGtrans
    |-- DIF
        |-- models
            |-- knife
            |-- bottle
                |-- checkpoints
                    |--model.pth

Copy all folders in '02876657'(bottle) to datasets, and file '03624134'(knife) as above. And arrange the files as follows:

|-- FGtrans
    |-- datasets
        |-- obj
            |-- knife
            |-- bottle
                |-- files

(*Here we take **bottle** as an example.*)

```shell
# Preprocess data, run:
python processdata/preprocess_ShapeNet.py --category bottle

# Generate sdf values, run:
python processdata/generate_sdf.py --category bottle --mode train
python processdata/generate_sdf.py --category bottle --mode eval
```

Training--Learning Categorical Dense Correspondence

To train DIF, run the following commands.

python DIF/train.py \
--config DIF/configs/train/bottle.yml

To transfer touch code, run the following commands. Please modify the path of the split in the config file, such as "split/eval/batch. txt" to "split/train/batch. txt" to generate transfer data for all objects.
```
python DIF/evaluate.py \
--config DIF/configs/eval/bottle.yml
```

Post processing of generated data

Convert the touch code to the original data.

python processdata/change_to_real_obj.py --category bottle --mode train
python processdata/change_to_real_obj.py --category bottle --mode eval

python processdata/remove_redundancy.py --category bottle

Acknowledgment

This repo is based on DIF-Net. Many thanks for their excellent works.

Citation

@article{wu2023functional,
  title={Functional Grasp Transfer Across a Category of Objects From Only one Labeled Instance},
  author={Wu, Rina and Zhu, Tianqiang and Peng, Wanli and Hang, Jinglue and Sun, Yi},
  journal={IEEE Robotics and Automation Letters},
  volume={8},
  number={5},
  pages={2748--2755},
  year={2023},
  publisher={IEEE}
}

License

Our code is released under MIT License.

wurina-github / FGTrans