This repository contains deep learning frameworks that we collected and ported to Keras. We wrapped those models into separate modules that aim to provide their functionality to users within 3 lines of code.
- Face detection:
- S3FD model ported from 1adrianb/face-alignment.
- MTCNN model ported from davidsandberg/facenet.
- Face landmarks detection:
- 2DFAN-4, 2DFAN-2, and 2DFAN-1 models ported from 1adrianb/face-alignment.
- Face parsing:
- BiSeNet model ported from zllrunning/face-parsing.PyTorch.
- Eye region landmarks detection:
- ELG model is ported from swook/GazeML.
*Each module follows the license of their source repo. Notice that some models were trained on dataset with non-commercial license.
(Please run
pip install keras==2.2.4
before initializaing models.)
This colab demo requires a GPU instance. It demonstrates all face analysis functionalities above.
models.detector.face_detector.FaceAlignmentDetector(fd_weights_path=..., lmd_weights_path=..., fd_type="s3fd")
Arguments
fd_weights_path
: A string. Path to weights file of the face detector model.lmd_weights_path
: A string. Path to weights file of the landmarks detector model.fd_type
: A string. Face detector backbone model of eithers3fd
ormtcnn
.
Example
from models.detector import face_detector
im = cv2.imread(PATH_TO_IMAGE)[..., ::-1]
fd = face_detector.FaceAlignmentDetector()
bboxes = fd.detect_face(im, with_landmarks=False)
The default model is 2DFAN-4. Lite models of 2DFAN-1 and 2DFAN-2 are also provided.
GPU | 2DFAN-1 | 2DFAN-2 | 2DFAN-4 |
---|---|---|---|
K80 | 74.3ms | 92.2ms | 133ms |
Example
from models.detector import face_detector
im = cv2.imread(PATH_TO_IMAGE)[..., ::-1]
fd = face_detector.FaceAlignmentDetector()
bboxes, landmarks = fd.detect_face(im, with_landmarks=True)
models.parser.face_parser.FaceParser(path_bisenet_weights=...)
Arguments
path_bisenet_weights
: A string. Path to weights file of the model.
Example
from models.parser import face_parser
im = cv2.imread(PATH_TO_IMAGE)[..., ::-1]
fp = face_parser.FaceParser()
# fp.set_detector(fd) # fd = face_detector.FaceAlignmentDetector()
parsing_map = fp.parse_face(im, bounding_box=None, with_detection=False)
models.detector.iris_detector.IrisDetector()
Faster face detection using MTCNN can be found in this repo.
Example
from models.detector import iris_detector
im = cv2.imread(PATH_TO_IMAGE)[..., ::-1]
idet = iris_detector.IrisDetector()
idet.set_detector(fd) # fd = face_detector.FaceAlignmentDetector()
eye_landmarks = idet.detect_iris(im)
models.verifier.face_verifier.FaceVerifier(extractor="facenet", classes=512)
Argument
extractor
: A string, one offacenet
,insightface
,ir50_ms1m
, orir50_asia
.classes
: An integer. Dimension of output embeddings.
Example
from models.verifier import face_verifier
im1 = cv2.imread(PATH_TO_IMAGE1)[..., ::-1]
im2 = cv2.imread(PATH_TO_IMAGE2)[..., ::-1]
fv = face_verifier.FaceVerifier(extractor="facenet")
# fv.set_detector(fd) # fd = face_detector.FaceAlignmentDetector()
result, distance = fv.verify(im1, im2, threshold=0.5, with_detection=False, with_alignment=False, return_distance=True)
models.estimator.gender_age_estimator.GenderAgeEstimator(model_type="insightface")
Arguments
model_type
: A string, onlyinsightface
is supported now.
Example
from models.estimator import gender_age_estimator
im = cv2.imread(PATH_TO_IMAGE)[..., ::-1]
gae = gender_age_estimator.GenderAgeEstimator()
gae.set_detector(fd) # fd = face_detector.FaceAlignmentDetector()
gender, age = gae.predict_gender_age(im, with_detection=True)
- insightface model, this file should be put in
./models/verifier/insightface/
. - IR50-MS-Celeb-1M model, this file should be put in
./models/verifier/face_evoLVe_ir50/
. - IR50-Asia-face model, this file should be put in
./models/verifier/face_evoLVe_ir50/
.
It works fine on Colab at this point (2019/06/11) but for certain Keras/TensorFlow version, it throws errors loading 2DFAN-1_keras.h5
or 2DFAN-2_keras.h5
.
- Keras 2.2.4
- TensorFlow 1.12.0 or 1.13.1
We learnt a lot from 1adrianb/face-alignment, zllrunning/face-parsing.PyTorch, swook/GazeML, deepinsight/insightface, davidsandberg/facenet, and ZhaoJ9014/face.evoLVe.PyTorch.