ASL Hand Gestures Recognition from the Webcam

This repository is an implementation of using CNN network to identify American Sign Language hand gestures from the webcam and show the output on the screen.

Demo:

Requirements

opencv-python
numpy
tensorflow
spellchecker

Data Set

Data set was created by me (approximately 1200 images for 1 gesture).

Data set structure:

|-- img_dataset
|   |-- train # train images ~80%
|       |-- A
|           |-- 0.jpg
|           |-- 1.jpg
|           |-- ...
|       |-- ...
|   |-- test # test images ~20%
|       |-- A
|           |-- 0.jpg
|           |-- 1.jpg
|           |-- ...
|       |-- ...
|   |-- orig_sample # RGB images of ASL gesture (one image for a gesture)
|       |-- A.jpg
|       |-- B.jpg
|       |-- ...

American Sign Language gestures:

Model Training

For classification model Canny Edge Detector was applied on images.

Images that are used for model:

CNN model performance:

Google Colaboratory Notebook with training process.

Run the Script

You might need to install the Git Large File Storage to be able to clone the repository.

$ git lfs install
$ git clone https://github.com/ruslan-kl/asl_recognition.git
$ cd asl_recognition
$ pip install -r requirements.txt
$ python asl_recognizer.py

Some instructions:

Once started adjust the threshold values for edge detection so you can see just the edges of your palm and fingers.
Press S to start/pause the output generation.
Press D to erase the output section.
Press Q to quit the script.
del, space and nothing do what they suppose to do.
Input double space to apply spellchecker on the last word.

Disclaimer

This project is just a way to practice my knowledge in CV and CNN in one place. It was not meant to be an app for actual practical usage.

rklymentiev / asl_recognition