Clear Face is python project with C++/Python libraries for facial landmark detection(in 2d and 3d), tracking faces and multiple models detection from faces such as:
- Gender, Expressions, Illumination, Pose, Occlusion, Age, and Makeup.
Download Repo
git clone https://github.com/DiaaZiada/ClearFace.gitcd ClearFaceVirtual Environment
# create virtual environment
python -m venv ./venv# activate virtual environment
source venv/bin/activateInstall Requirements
pip install -r requirements.txtDownload libtorch for c++ (CPU)
./download.shnote: this command will download and extract the libarary
Build all c++ files
./build.shto use ClearFace execute run.py file with various options
usage: run.py [-h] [--show] [--_2d] [--_3d] [--_2d3d] [--tracking]
[--delay DELAY] [--inputs_path INPUTS_PATH]
[--video_name VIDEO_NAME] [--outputs_path OUTPUTS_PATH]
[--models_path MODELS_PATH] [--cam_num CAM_NUM]
[--models MODELS [MODELS ...]]
Clear Face is python project with C++/Python libraries for facial landmark detection(in 2d and 3d), tracking faces and multiple models detection from faces such as:
- Gender, Expressions, Illumination, Pose, Occlusion, Age, and Makeup.
optional arguments:
-h, --help show this help message and exit
--show set this parameter to True value if you want display
images/videos while processing, default is False
--_2d set this parameter to True value if you 2d landmarks
from from the 2d model, default is False
--_3d set this parameter to True value if you 3d landmarks
from from the 3d model, default is False
--_2d3d set this parameter to True value if you 2d landmarks
from from the 3d model, default is False
--tracking set this parameter to True value if you want tracking
faces in images/videos, default is False
--delay DELAY amount of seconds to wait to switch between images
while show the precess
--inputs_path INPUTS_PATH
path for directory contains images/videos to process,
if you don't use it webcam will open to start the
record
--video_name VIDEO_NAME
name of recorded video from camera
--outputs_path OUTPUTS_PATH
path for directory to add the precesses images/videos
on it, if you don't use it output directory will
created and add the precesses images/videos on it
--models_path MODELS_PATH
path for directory contains pytorch model
--cam_num CAM_NUM number of camera to use it
--models MODELS [MODELS ...]
first index refers to gender model, second index
refers to expression model, and third index refers to
multiple models
Examples
for process images/videos in directory
$python run.py --inputs_path /path/to/inputs_dir --tracking --show --_2d3d --_3d
to use webcam
$python run.py ---tracking --show --_2d3d --_3d
- Gender : Male, Female
- Expressions : Anger, Happiness, Sadness, Surprise, Fear, Disgust
- Illumination : Bad, Medium, High
- Pose : Frontal, Left, Right, Up, Down
- Occlusion : Glasses, Beard, Ornaments, Hair, Hand, None, Others
- Age : Child, Young, Middle and Old
- Makeup : Partial makeup, Over-makeup
models in this project are based on Real-time Convolutional Neural Networks for Emotion and Gender Classification paper model architecture:
this part of the project consist of 3 models:
- Gender
- Expression
- Multiple
trained done by using IMDB , WIKI, IMFDB datasets. consist of ~ 600 K image, and by using tencrop data augmentation dataset increased to be ~ 6 M image Accuracy ~ 78% using only 6 epochs and it will reach higher accuracy expected to be ~ 96 %
trained done by using FER , IMFDB, JAFFE image datasets. consist of ~ 40 K image, and by using tencrop data augmentation dataset increased to be ~ 400 K image Accuracy ~ 60% using 15 epochs and it will reach higher accuracy expected to be ~ 66 %
this model is little bit different form Gender & Expression models in this model we use one feature extractor model and 5 different classifiers each classifier predict specific features form faces and they are:
- Illumination : Bad, Medium, High
- Pose : Frontal, Left, Right, Up, Down
- Occlusion : Glasses, Beard, Ornaments, Hair, Hand, None, Others
- Age : Child, Young, Middle and Old
- Makeup : Partial makeup, Over-makeup
trained done by using IMFDB image datasets consist of ~ 3 K image, and by using tencrop data augmentation dataset increased to be ~ 30 K image Accuracy ~ 77% using 15 epochs
all training process done on notebook using Google Colab cloud
we convert ower models to torchscript to load them in c++ and use them as extention of ClearFace all convertions about this part will found in this notebook
we used the centroid tracking algorithm
kaggle youtube faces with facial keypoints ~ 150K image
all training process for the 2d and 3d model done in this notebook using Google Colab cloud
You can email me on diaa.elsayedziada@gmail.com to get the pre-trained models
- Real-time Convolutional Neural Networks for Emotion and Gender Classification paper
- Simple object tracking with OpenCV by pyimagesearch
- Extending TorchScript with Custom C++ Operators
- CMake Tutorial
- Ryan Reynolds and Jake Gyllenhaal interview for LIFE, DEADPOOL - UNCENSORED Youtube
- 1adrianb face alignment repo