Handsign is a simple AI-based hand gesture recognition that translates a hand pose into the American Sign Language (ASL) alphabet. Using Tensorflow JS and its Handpose preloaded model to detect the hand object and its parts. Handsign also uses an additional library called Fingerpose to classify certain of custom hand gestures based on the finger position.
#dohackathon #madewithTFJS
- Clone the repository
# copy the repo to your machine
git clone https://github.com/syauqy/handsign-tensorflow-gatsby.git- Start the project
# move to the project folder and install all dependencies
cd handsign-tensorflow-gatsby
yarn install- Run the project on your local machine
# run Gatsby
yarn develop- The project is live 🚀
Your project is live and running at http://localhost:8000
You can edit the core program at src/pages/app.js
uncomment the <pre> component
<Image h="150px" objectFit="cover" id='emojimage'/>
// uncomment this
{/* <pre className="pose-data" color="white" style={{position: 'fixed', top: '150px', left: '10px'}} >Pose data</pre> */}
</Container>uncomment the estimatedGestures data to change '.pose-data' innerHTML
// document.querySelector('.pose-data').innerHTML =JSON.stringify(estimatedGestures.poseData, null, 2);the estimatedGestures data will render on your screen.
-
Tensorflow JS - A Library for ML in JS.
-
Handpose - A lightweight ML pipeline consisting of two models: A palm detector and a hand-skeleton finger tracking model.
-
Fingerpose - A pose classifier for hand landmarks detected by TensorFlow.js Handpose's model.
-
Sign language illustration is created by Pelin Kahraman
If you want to learn more about Tensorflow JS and custom gesture handpose, please kindly check these amazing videos
-
Machine Learning with TensorFlow in JavaScript - by Jason Lengstorf and Jason Mayes
-
Building a Real Time Sign Language Detection App with React.JS and Tensorflow.JS - by Nicholas Renotte
