This repository contains a customized tool for automatically detecting and annotating 3D hand joint coordinates based on OpenPose models as well as training and testing scripts for a Convolutional Bi-LSTM model for performing 3D hand pose estimation.
A custom tool based on OpenPose hand detection models for detecting 2D keypoints and automatically triangulating 3D keypoints is included in this repository under the multicam folder. The tool was built upon Intel RealSense APIs and has been customized for Intel RealSense D415 series cameras. The full documentation can be found here.
The training and testing scripts for the C-Bi-LSTM model can be found in the main repository folder for the ICVL, NYU and MSRA datasets: icvl_basic.py, nyu_basic, msra_basic.py. There is also a realtime pipeline script that is configured to work with an Intel RealSense D415 series camera: icvl_realtime_pipeline.py. The hand centers computed by V2V-PoseNet can also be found here)
Overview of C-Bi-LSTM model:
Demo of the model in realtime deployment:
Sample predictions on ICVL and NYU dataset:
