AI4Bharat / INCLUDE

Code for INCLUDE paper with pre-trained models

INLCUDE - Isolated Indian Sign Language Recognition

This repository contains code for training models on INCLUDE dataset

Dependencies

Install the dependencies through the following command

>> pip install -r requirements.txt

Steps

• Download the INCLUDE dataset
• Run generate_keypoints.py to save keypoints from Mediapipe Hands and Blazepose for train, validation and test videos.

>> python generate_keypoints.py --include_dir <path to downloaded dataset> --save_dir <path to save dir> --dataset <include/include50>

• Run runner.py to train a machine learning model on the dataset

>> python runner.py --dataset <include/include50> --use_augs --model transformer --data_dir <location to saved keypoints>

• Use the --use_pretrained flag to either perform only inference using pretrained model or resume training with the pretrained model.

>> python runner.py --dataset <include/include50> --use_augs --model transformer --data_dir <location to saved keypoints> --use_pretrained <evaluate/resume_training>

• To get predictions for videos from a pretrained model, run the following command.

>> python evaluate.py --data_dir <dir with videos>

Citation

@inproceedings{10.1145/3394171.3413528,
author = {Sridhar, Advaith and Ganesan, Rohith Gandhi and Kumar, Pratyush and Khapra, Mitesh},
title = {INCLUDE: A Large Scale Dataset for Indian Sign Language Recognition},
year = {2020},
isbn = {9781450379885},
publisher = {Association for Computing Machinery},
doi = {10.1145/3394171.3413528},
numpages = {10},
series = {MM '20}
}