MASKED-FACE-RECOGNITION-OR-PERIOCULAR-ARTIFICIAL-INTELLIGENCE-APP

WORLD'S FIRST OPEN SOURCE MASKED FACE RECOGNITION ARTIFICIAL INTELLIGENCE APP

Author: God Bennett (My legal name was changed from Jordan to God.)

I could not find any artificial intelligence application that did masked face recognition, so I had decided to create an ai based solution, last year in October 2020. The solution took about 5 hours to implement and now the code is available here.

This artificial intelligence project is a masked face recognition (or periocular recognition) artificial intelligence application, built atop this original repository, that does not do masked face recognition, as it only does recognition of faces without masks, and also, I added some neccessary fixes as the original repository did not run without said fixes.

Apart from not running due to artificial intelligence related errors (whose fixes I addressed in section "C" below), it also did not implement a graphical user interface.

My project here resolves all of the aforementioned issues and constraints.

Click on animated image below to zoom!

Example use case

Identify/recognize previously flagged people in a video in an ATM wearing a mask.

[A] Instructions_User

Download the related face recognition model, and copy the .t7 file to this repository's directory on your computer.
Follow [B] God's installation modules.
To evaluate/test on model trained on God's face/masked face, run my LAUNCH_APP.bat file to evaluate "_test!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!/bennett_god_clip_masked.mp4" or "_test/bennett_god_clip_unmasked.mp4" or "_test!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!/holness_testing_sample_1.mp4".
Press "q" to end processing of the popup video processing window, so a next video can be loaded by the main Ui.
Ui mode for training model coming soon.

[A] Instructions_Developer

i. Original repository, that has errors addressed by myself in (ii) below:

https://github.com/aakashjhawar/face-recognition-using-deep-learning

ii. God's fixed version of the base code (that didn't do masked face recognition or had a gui), is documented in section C: "God's fixes/modifications/additions".

This includes modfied versions of recognize_video.py, train_model.py without which several errors would occur.
I also included new files for a simple gui, as seen in "recognize_video_v3_ui.py" etc.

iii. Follow my installation modules guide [B], and use my files from (ii) above in this repository (ii) above because the default code (i) failed to run without fixes in section [C] as seen in the files in this repository.

Note: ~50 masked and ~20 unmasked images were used for training for the "God" identity.

Usage: Ensure that custom training data is different from inference samples.

[B] God's installation modules

Windows

python 3.5 base (as advised by default repo)

scikit_learn 0.19.2 (extracted by God from recognizer.pickle, line 174 <NsS'_sklearn_version'>)

scipy 1.0.0 (extracted by God from scikit_learn webpage suggested version minimum) https://scikit-learn.org/0.19/whats_new.html

numpy 1.16.4 (installed by aiming to resolve no numpy.testing.decorators error)

If you want to use examples in /GOD DATA + COMMANDS:
- Download Masked_Face_Recognition_Dataset_by_God.zip.
- Copy image contents of Masked_Face_Recognition_Dataset_by_God.zip folder, to "dataset" folder in this repository's directory.
- Change all Python35 path locations in all batch files to yours.
- Change all .py locations in all batch files to your specific locations.

[C] God's fixes/modifications/additions

Notes about God's repairs to default repo:

--machine learning fixes:

Pickle errors removed from recognizer_image.py from default py files.
SVC Load errors from default py files fixed by dumping sklearn model in train_model.py, and loading same model in recognize_image.py
Code added to trigger processing of video files instead of live stream.
Code added to stamp picture of recognized suspect, in new overview_recognized folder.

--basic fixes:

Basic python print bracket errors removed from default py files.
Tab errors removed from default py files.

[D] God's guidelines for setting up training data (for both masked and unmasked recognition):

1.a Take a 10 second clip of person with face masked. (Using [ii] from Suggested Tools below, if applicable)

1.b Convert clip to collection of images (using [i] from Suggested Tools below), and follow instructions to place images in dataset folder from section [A].

2.a Take a 10 second clip of person with face unmasked. (Using [ii] from Suggested Tools below, if applicable)

2.b Convert clip to collection of images (using [i] from Suggested Tools below), and follow instructions to place images in dataset folder from section [A].

Note, place all images taken from mp4 to image set conversion above, pertaining to one person, in the same person folder from [A].

Suggested Tools:

i. Mp4 to jpg collection (download resulting zip): https://ezgif.com/video-to-jpg

ii. WebCam recorder: https://webcamera.io/

Suggested Training Data

See the famous Masked Dataset paper, that unfortunately doesn't provide a solution/code, but it does provide gigabytes of data for training.

Neither of my solutions use this currently, and both could reasonably benefit from the aforementioned training set. There are also solutions for creating masked data available across the web.

JordanMicahBennett / MASKED-FACE-RECOGNITION-OR-PERIOCULAR-ARTIFICIAL-INTELLIGENCE-APP