JS-RML / dig-grasping

This package presents an autonomous system for singulating and picking thin profile objects, for example, Go stones and capsules, from dense clutter.

Dig-grasping

1. Overview

This package presents an autonomous system for singulating and picking thin profile objects, for example, Go stones, capsules and domino blocks, from dense clutter. It is an implementation of Dig-Grasping, a new robotic manipulation technique for simultaneously singulating and picking objects from clutter, by leveraging planar quasistatic pushing as a way of direct physical interaction between the object to pick and the robot. As will be provided in this package, dig-grasping exhibits a gripper design with digit asymmetry, realized as a two-fingered gripper with finger length differences. Beyond picking, dig-grasping also presents more complex tasks such as autonomous pick-and-place of Go stones and pick-and-pack of capsules, which are included in this package.

Full Video can be seen from this link.

2. Prerequisites

2.1 Hardware

• Universal Robot UR10
• Robotiq 140mm Adaptive parallel-jaw gripper
• RealSense Camera SR300
• Customized Finger design features fingertip concavity
• Extendable Finger for realizing finger length differences during digging
• Extendable Palm developed in another package: Shallow-Depth Insertion (Dexterous Ungrasping). See the link for detailed explanation

2.2 Software

This implementation requires the following dependencies (tested on Ubuntu 16.04 LTS):

• ROS Kinetic
• Urx for UR10 robot control
• robotiq_2finger_grippers: ROS driver for Robotiq Adaptive Grippers
• Mask R-CNN for instance segmentation (also see the dependencies therein). Also download the trained weights for Go stone and capsule.
• OpenCV 3.4.1 and Open3D 0.7.0.0

Note: The online compiler Jupyter Notebook is needed to run our program.

3. Planar Quasistatic Pushing Simulator

The simulator describes the finger-object interaction where the object is pushed by a position-controlled finger along a straight line. It is implemented in MATLAB. Right now the simulator is demoed with an elliptical or a rectangular object. Given initial pushing conditions - a initial contact position and an orientation of the line of pushing, the simulator will return the trace of the object being pushed, seen from an observer moving together with the finger. For more details, please see the references [1], [2].

To run the simulator, enter the folder pushing_simulator/elli or pushing_simulator/rect,

Run

push_elli_demo.m

or

push_rect_demo.m

Simulation Parameters

• a: length of the object
• b: width of the object
• N: gravity of the object
• mu_s: friction coefficient between the object and the supporting surface
• mu_c: friction coefficient between the object and the pushing finger
• psi: orientation of the line of pushing
• d: initial contact position

An example

a=0.046, b=0.012, N=0.02; mu_s=0.2, mu_c=0.8, psi=120, d=0.006

4. A Quick Start of Real Experiments

Create your catkin workspace:

cd ~/catkin_ws/src
git clone https://github.com/HKUST-RML/Dig-grasping.git
cd ..
catkin_make

Activate robotiq 2-fingered gripper and RealSense Camera in two different terminals:

roslaunch dig-grasping gripper.launch sim:=true
roslaunch realsense2_camera rs_camera.launch align_depth:=true 

Picking

For picking Go stones

1. Open another terminal, start Jupyter Notebook via jupyter notebook, and run instance_segmentation.ipynb for instance segmentation and object pose detection.

2. Start another Jupyter Notebook in a new terminal, and run Go_stone_pick.ipynb.

For picking capsules

1. run instance_segmentation_capsules.ipynb
2. run Capsule_pick.ipynb

For picking domino blocks

1. run Domino_detection.ipynb
2. run Domino_pick.ipynb

Placing

Open a terminal,

cd scripts/dexterous_ungrasping/script

For Go stone: python Go_stone_place.py

For capsule: python Capsule_place.py

Maintenance

For any technical issues, please contact: Zhekai Tong ([e-mail]), and Yu Hin Ng ([e-mail]).