heyoh-app / gestures-detector

Heyoh is an open-source virtual camera based on PyTorch. It is a training and inference code for heyoh-camera app.

Heyoh Object Detection / Tracking pipeline

Heyoh is an open source virtual camera based on PyTorch. It is a training and inference code for heyoh-camera app. Github project page.

Contributors: Marko Kostiv, Danylo Bondar, Oleh Sehelin, Ksenia Demska

PyTorch Annual Hackathon 2021

Table of Contents

1. Prerequisites
2. Getting started
3. Training
4. Usage
5. Supplementary materials

Prerequisites

• Linux or macOS
• CUDA 11.1
• git-lfs

Getting started

1. Create and activate conda environment:

conda create -n heyoh python=3.8 -y
conda activate heyoh

2. Install requirements:

pip3 install -r multitask_lightning/requirements.txt

Training

1. Navigate to multitask_lightning directory

cd multitask_lightning/

2. Modify config file if needed:

nano configs/train_config.yaml

3. Run training:

python train.py configs/train_config.yaml

Usage UPDATED FOR APPLE M1 SUPPORT

We developed a python debug application which processes frames similarly to heyoh-camera app in Python. It is extremely useful for a faster prototyping and hyperparameters tuning.

HOW TO TEST

Guide on how to convert pytorch_lightning checkpoint to TorchScript and CoreML models and run the debug application.

Supplementary materials

• Sample dataset description
• Pre-trained checkpoint