Awesome Touch

Owing to my personal interest, this collection may focus more on vision-based tactile sensors and learning-based applications. Keep updating! ! !

Sensors
Applications
Software
Review
Thesis
Products
Open Source

Sensors

For New Researchers

If you are a new researcher in vision-based tactile sensing, you may begin with these papers because they propse sensors with quite different principles.

Measurement of force vector field of robotic finger using vision-based haptic sensor, Sato et al., IROS 2008
Combining finger vision and optical tactile sensing: Reducing and handling errors while cutting vegetables, Yamaguchi et al., Humanoids 2016
GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force, Yuan et al., MDPI Sensors 2017
The TacTip Family: Soft Optical Tactile Sensors with 3D-Printed Biomimetic Morphologies, Ward-Cherrier et al., SoRo 2018
Soft-bubble: A highly compliant dense geometry tactile sensor for robot manipulation, Alspach et al., RoboSoft 2019
Design, Motivation and Evaluation of a Full-Resolution Optical Tactile Sensor, Sferrazza et al., MDPI Sensors 2019
In-Hand Object Localization Using a Novel High-Resolution Visuotactile Sensor, Cui et al., Trans. Ind. Electron. 2022
DelTact: A Vision-based Tactile Sensor Using Dense Color Pattern, Zhang et al., RAL 2022
DTact: A Vision-Based Tactile Sensor that Measures High-Resolution 3D Geometry Directly from Darkness, Lin et al., ICRA 2023

Applications

Manipulation/Grasping

Localization and manipulation of small parts using GelSight tactile sensing, Li et al., IROS 2014
Tactile-Based Insertion for Dense Box-Packing, Dong et al., IROS 2019
Maintaining Grasps within Slipping Bounds by Monitoring Incipient Slip, Dong et al., ICRA 2019
Manipulation by feel: Touch-based control with deep predictive models, Tian et al., ICRA 2019
Sensing Shear Forces During Food Manipulation: Resolving the Trade-Off Between Range and Sensitivity, Song et al., ICRA 2019
Building a Library of Tactile Skills Based on FingerVision, Belousov et al., Humanoids 2019
Self-Attention Based Visual-Tactile Fusion Learning for Predicting Grasp Outcomes, Cui et al., RAL 2020
Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard, Church et al., RAL 2020
Swingbot: Learning physical features from in-hand tac?tile exploration for dynamic swing-up manipulation, Wang et al., IROS 2020
Tactile Dexterity: Manipulation Primitives with Tactile Feedback, Hogan et al., ICRA 2020
Cable manipulation with a tactile-reactive gripper, She et al., IJRR 2021
A Robust Controller for Stable 3D Pinching Using Tactile Sensing, Psomopoulou et al., RAL 2021
Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry, Dong et al., ICRA 2021
Improving Grasp Stability with Rotation Measurement from Tactile Sensing, Kolamuri et al., IROS 2021
Dynamic Modeling of Hand-Object Interactions via Tactile Sensing, Zhang et al., IROS 2021
SEED: Series Elastic End Effectors in 6D for Visuotactile Tool Use, Suh et al., arxiv 2021
Meta-Residual Policy Learning: Zero-Trial Robot Skill Adaptation via Knowledge Fusion, Hao et al., RAL 2022
Active Visuo-Tactile Interactive Robotic Perception for Accurate Object Pose Estimation in Dense Clutter, Murali et al., RAL 2022
Autonomous Learning of Page Flipping Movements via Tactile Feedback, Zheng et al., TRO 2022
Where Shall I Touch? Vision-Guided Tactile Poking for Transparent Object Grasping, Jiang et al., Transactions on Mechatronics 2022
Efficient Tactile Simulation with Differentiability for Robotic Manipulation, Xu et al., CoRL 2022
Tactile Pose Estimation and Policy Learning for Unknown Object Manipulation, Kelestemur et al., arxiv 2022
Visuo-Tactile Transformers for Manipulation, Chen et al., arxiv 2022
Grasp Stability Prediction with Sim-to-Real Transfer from Tactile Sensing, Si et al., arxiv 2022
Safely Learning Visuo-Tactile Feedback Policies in Real For Industrial Insertion, Fu et al., arxiv 2022
Manipulation via Membranes: High-Resolution and Highly Deformable Tactile Sensing and Control, Oller et al., arxiv 2022
Learning Self-Supervised Representations from Vision and Touch for Active Sliding Perception of Deformable Surfaces, Kerr et al., arxiv 2022
Active Extrinsic Contact Sensing: Application to General Peg-in-Hole Insertion, Kim et al., ICRA 2022
Task-Driven In-Hand Manipulation of Unknown Objects with Tactile Sensing, Pan et al., arxiv 2022
Visual-tactile Fusion for Transparent Object Grasping in Complex Backgrounds, Li et al., arxiv 2022
Integrating High-Resolution Tactile Sensing into Grasp Stability Prediction, Chumbley et al., CRV 2022
Skill generalization of tubular object manipulation with tactile sensing and Sim2Real learning, Zhao et al., Robotics and Autonomous Systems 2023
Learning height for top-down grasps with the DIGIT sensor, Bernardi et al., ICRA 2023

Classification/Recognition

Majority Voting: Material Classification by Tactile Sensing Using Surface Texture, Jamali et al., TRO 2011
Sensing and recognizing surface textures using a gelsight sensor, Li et al., CVPR 2013
Connecting Look and Feel: Associating the visual and tactile properties of physical materials, Yuan et al., CVPR 2017
Shape-independent hardness estimation using deep learning and a GelSight tactile sensor, Yuan et al., ICRA 2017
Vitac: Feature sharing between vision and tactile sensing for cloth texture recognition, Yuan et al., ICRA 2018
Active clothing material perception using tactile sensing and deep learning, Yuan et al., ICRA 2018
Understanding Dynamic Tactile Sensing for Liquid Property Estimation, Huang et al., arxiv 2022
Visuotactile Affordances for Cloth Manipulation with Local Control, Sunil et al., arxiv 2022
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation, Li et al., arxiv 2022

Mapping/Localization

Tracking objects with point clouds from vision and touch, Izatt et al., ICRA 2017
3D Shape Perception from Monocular Vision, Touch, and Shape Priors, Wang et al., IROS 2018
End-to-end pixelwise surface normal estimation with convolutional neural networks and shape reconstruction using GelSight sensor, Li et al., ROBIO 2018
From Pixels to Percepts: Highly Robust Edge Perception and Contour Following Using Deep Learning and an Optical Biomimetic Tactile Sensor, Lepora et al., RAL 2019
Convolutional Autoencoder for Feature Extraction in Tactile Sensing, Polic et al., RAL 2019
Tactile Mapping and Localization from High-Resolution Tactile Imprints, Bauza et al., ICRA 2019
Fast Model-Based Contact Patch and Pose Estimation for Highly Deformable Dense-Geometry Tactile Sensors, Kuppuswamy et al., RAL 2020
Deep Gated Multi-modal Learning: In-hand Object Pose Changes Estimation using Tactile and Image Data, Anzai et al., IROS 2020
3D Shape Reconstruction from Vision and Touch, Smith et al., NeurIPS 2020
Tactile SLAM: Real-time inference of shape and pose from planar pushing, Suresh et al., ICRA 2021
Tactile Object Pose Estimation from the First Touch with Geometric Contact Rendering, Bauza et al., CoRL 2021
Active 3D Shape Reconstruction from Vision and Touch, Smith et al., NeurIPS 2021
Active Visuo-Tactile Point Cloud Registration for Accurate Pose Estimation of Objects in an Unknown Workspace , Murali et al., arxiv 2021
Tac2Pose: Tactile Object Pose Estimation from the First Touch, Bauza et al., arxiv 2022
ShapeMap 3-D: Efficient shape mapping through dense touch and vision, Suresh et al., ICRA 2022
PatchGraph: In-hand tactile tracking with learned surface normals, Sodhi et al., ICRA 2022
MidasTouch: Monte-Carlo inference over distributions across sliding touch, Suresh et al., CoRL 2022
Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensing, Xu et al., arxiv 2022
Using Collocated Vision and Tactile Sensors for Visual Servoing and Localization, Chaudhury et al., arxiv 2022
Simultaneous Contact Location and Object Pose Estimation Using Proprioception and Tactile Feedback, Sipos et al., arxiv 2022
VisuoTactile 6D Pose Estimation of an In-Hand Object Using Vision and Tactile Sensor Data, Dikhale et al., RAL 2022
In-Hand Pose Estimation Using Hand-Mounted RGB Cameras and Visuotactile Sensors, Gao et al., IEEE 2023
Tac2Structure: Object Surface Reconstruction Only Through Multi Times Touch, Lu et al., RAL 2023
Tac-VGNN: A Voronoi Graph Neural Network for Pose-Based Tactile Servoing, Fan et al., arxiv 2023

Extrinsic Contact Sensing

Extrinsic Contact Sensing with Relative-Motion Tracking from Distributed Tactile Measurements, Ma et al., ICRA 2021
Neural Contact Fields: Tracking Extrinsic Contact with Tactile Sensing, Higuera et al., arxiv 2022
Simultaneous Tactile Estimation and Control of Extrinsic Contact, Kim et al., ICRA 2023

Others

Exploiting Distributed Tactile Sensors to Drive a Robot Arm Through Obstacles, Albini et al., RAL 2021
Extended Tactile Perception: Vibration Sensing through Tools and Grasped Objects, Taunyazov et al., IROS 2021
MidasTouch: Monte-Carlo inference over distributions across sliding touch, Suresh et al., CoRL 2022
SLURP! Spectroscopy of Liquids Using Robot Pre-Touch Sensing, Hanson et al., arxiv 2022
Touching a NeRF: Leveraging Neural Radiance Fields for Tactile Sensory Data Generation, Zhong et al., CoRL 2022
Learning to Synthesize Volumetric Meshes from Vision-based Tactile Imprints, Zhu et al., arxiv 2022
Going In Blind: Object Motion Classification using Distributed Tactile Sensing for Safe Reaching in Clutter, Thomasson et al., arxiv 2022
Touch and Go: Learning from Human-Collected Vision and Touch, Yang et al., NeurIPS 2022
Development and Evaluation of a Learning-based Model for Real-time Haptic Texture Rendering, Heravi et al., arxiv 2022

Software

Simulator

Generation of GelSight Tactile Images for Sim2Real Learning, Gomes et al., RAL 2021
Simulation of Vision-based Tactile Sensors using Physics based Rendering, Agarwal et al., ICRA 2021
Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation, Church et al., CoRL 2021
Tactile Gym 2.0: Sim-to-Real Deep Reinforcement Learning for Comparing Low-Cost High-Resolution Robot Touch, Lin et al., RAL 2022
TACTO: A Fast, Flexible, and Open-Source Simulator for High-Resolution Vision-Based Tactile Sensors, Wang et al., RAL 2022
Taxim: An Example-Based Simulation Model for GelSight Tactile Sensors, Si et al., RAL 2022
Bidirectional Sim-to-Real Transfer for GelSight Tactile Sensors With CycleGAN, Chen et al., RAL 2022
Learning the Dynamics of Compliant Tool-Environment Interaction for Visuo-Tactile Contact Servoing, Merwe et al., arxiv 2022
Efficient Tactile Simulation with Differentiability for Robotic Manipulation, Xu et al., CoRL 2022
Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors, Chen et al., RAL 2023

Library

PyTouch: A Machine Learning Library for Touch Processing, Lambeta et al., ICRA 2021

Review

Artificial Sense of Slip—A Review, Francomano et al., IEEE Sensors 2013
Tactile sensing in dexterous robot hands — Review, Kappassov et al., Robotics and Autonomous Systems 2015
Tactile Image Sensors Employing Camera: A Review, Shimonomura et al., MDPI Sensors 2019
Tactile Sensors for Advanced Intelligent Systems, Wang et al., Advanced Intelligent Systems 2019
Visuotactile Sensors With Emphasis on GelSight Sensor: A Review, C. Abad et al., IEEE Sensors 2020
On the Design and Development of Vision-based Tactile Sensors, Shah et al., JINT 2021
A survey of visuotactile sensing technologies for robotic manipulation, Cui et al., Chinese Journal of Intelligent Science and Technology 2022
Hardware Technology of Vision-Based Tactile Sensor: A Review, Zhang et al., IEEE Sensors 2022

Thesis

Tactile Measurement with a GelSight Sensor, Wenzhen Yuan, Master, 2014
Robust object pose estimation with point clouds from vision and touch, Gregory Russell, Master, 2017
3D shape perception from vision and touch, Shaoxiong Wang, Master, 2019
Vision-based proprioception of a soft robotic finger with tactile sensing, Sandra Q. Liu, Master, 2020
Discovering the patterns of human-environment interactions using scalable functional textiles, Yiyue Luo, Master, 2020
Deformable Object Manipulation with a Tactile Reactive Gripper, Neha Sunil, Master, 2021
Touching is believing : sensing and analyzing touch information with GelSight, Rui Li, Ph.D., 2015
High-resolution tactile sensing for robotic perception, Wenzhen Yuan, Ph.D., 2018
Reactive manipulation with contact models and tactile feedback, Francois R. Hogan, Ph.D., 2020
Data-driven Tactile Sensing using Spatially Overlapping Signals, Pedro Piacenza, Ph.D., 2020
High-resolution Tactile Sensing for Reactive Robotic Manipulation, Siyuan Dong, Ph.D., 2021
A general framework for high-resolution robotic tactile sensing: design, simulation, and learning, Carmelo Sferrazza, Ph.D., 2022

richardrl / Awesome-Touch

Awesome Touch

Table of Contents

Sensors

For New Researchers

Vision-Based

Visual-Tactile

Non-Vision-Based

Applications

Manipulation/Grasping

Classification/Recognition

Mapping/Localization

Extrinsic Contact Sensing

Others

Software

Simulator

Library

Review

Thesis

Products

Open-Source

Sensor

Sensor Support

Simulator/Library

Dataset

Algorithms

About