PhucNDA / FaceID--YOLOV5.ArcFace

ONNX implementation of YOLOv5 and Siamese Network (ResNet100) with ArcFace loss for Face Detection and Recognition

FaceID--YOLOV5.ArcFace

ONNX implementation of YOLOv5 and Siamese Network (ResNet100) with ArcFace loss for Face Detection and Recognition

face_recog_nhom.mp4

Environment Setup

conda install faceid torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia
conda activate faceid
pip install opencv-python
pip install onnxruntime==1.14.0
pip install onnxruntime-gpu==1.14.0
git clone https://github.com/PhucNDA/FaceID--YOLOV5.ArcFace
cd FaceID--YOLOV5.ArcFace

Ensuring the right data tree format

FaceID--YOLOV5.ArcFace
├── database_image
│   ├── profile1.png
|   ├── profile2.png
|   ├── profile3.png
|   ├── ...
├── database_tensor
│   ├── profile1.npy
|   ├── profile2.npy
|   ├── profile3.npy
|   ├── ...

database_image: containing image for each profile

database_tensor: containing vector feature extracted by pretrained backbone for each profile

Face Detection

Making a few key modifications to the YOLOv5 and optimize it for face detection. These modifications include adding a five-point landmark regression head, using a stem block at the input of the backbone, using smaller-size kernels in the SPP, and adding a P6 output in the PAN block. YOLOv5 is pretrained on WIDERFace datasets and we already converted it to the ONNX gpu runtime:

yolov5m-face.onnx

Adding New Face

Pre-trained backbone ResNet100 weights on Glint360K which contains 17091657 images of 360232 individuals is available at : weights

Manually add new face images to folder:

database_image

For fast precomputation, pre-extract database images to .npy tensor:

python feature_extraction.py --weight 'weights/backbone.pth' --path_database database_image

database_tensor

Convert backbone weight to ONNX implementation:

python converttoonnx.py

Face Recognition

It is implemented on ResNet100 backbone and SOTA ArcFace loss: paper

Following is the pipeline of ArcFace loss:

Transfer-learning by training new faces on custom datasets is not neccessary in face recognition algorithm. Using a pre-trained weight on large-scale dataset is enough for feature extraction backbone. Using asia-oriented dataset might lead to bias toward inference phase.

Webcam Real-time GPU inference

python detection_gpu.py

This version is good enough for face recognition system. Adding threshold for Unknown classification depends on user-experience. The model performs well on tilted face and obscured face (facemask). More improvements will be carried out in the future.

face_recog_phuc.mp4

Contributors

I want to express sincere thanks to my colleagues at University of Information Technology - UIT for their contribution to this project.

Serial	Full name	Github	Email
1	Nguyễn Đức Anh Phúc	PhucNDA	[e-mail]
2	Huỳnh Viết Tuấn Kiệt	hiimking1509	[e-mail]
3	Nguyễn Nhật Trường	truongnn	[e-mail]
4	Lại Chí Thiện	laichithien	[e-mail]
5	Lê Việt Thịnh	levietthinh	[e-mail]

Acknowledgement

This project is carried out in order to build and embedded system for door unlocking 10E8 at University of Information Technology - UIT. Further implementation on Raspberry pi 4 will not be updated for privacy concerns. The ONNX model can be used for academic and reproduction purposes only, but any commercial use is prohibited.