opencv-mtcnn
This is an inference implementation of MTCNN (Multi-task Cascaded Convolutional Network) to perform Face Detection and Alignment using OpenCV's DNN module.
MTCNN
[ZHANG2016] Zhang, K., Zhang, Z., Li, Z., and Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10):1499–1503.
https://kpzhang93.github.io/MTCNN_face_detection_alignment/paper/spl.pdf
OpenCV's DNN module
Since OpenCV 3.1 there is a module called DNN that provides the inference support. The module is capable of taking models & weights from various popular frameworks such as Caffe, tensorflow, darknet etc.
You can read more about it here - https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV
Note that at present there is no support to perform training in OpenCV's DNN module and if I understood correctly there is no intention either.
Compile / Run
Requirements
- OpenCV 3.4+
- Boost FileSystem (1.58+) [only required for the sample application]
- CMake 3.2+
I am using CMake as the build tool. Here are the steps to try the implementation -
# compiling the library and the sample application
git clone https://github.com/ksachdeva/opencv-mtcnn
cd opencv-mtcnn
mkdir build
cd build
cmake ..
cmake --build .
# running the sample application
cd build
./sample/app <path_to_models_dir> <path_to_test_image>
# here are some example cmd lines to run with the model and images in the test repository
# An image with 0 human faces (have picture of 4 dogs)
./sample/app ../data/models ../data/dogs.jpg
# An image with 1 face
./sample/app ../data/models ../data/Aaron_Peirsol_0003.jpg
# An image with 7 faces
./sample/app ../data/models ../data/2007_007763.jpg
Result
Here is an example of how the execution of the sample application looks like
Acknowledgments
Most of the implementations of MTCNN are based on either Caffe or Tensorflow. I wanted to play with OpenCV's DNN implementation and understand the paper bit better. While implementing it, I looked at various other C++ implementations (again all of them use Caffe) and more specifically borrowed utilities from https://github.com/golunovas/mtcnn-cpp. IMHO, I found his implementation (in C++) that is based on Caffe to be the cleanest amongst many others.
The model files are taken from https://github.com/kpzhang93/MTCNN_face_detection_alignment/tree/master/code
The image file "Aaron_Peirsol_0003.jpg" is from the LFW database (http://vis-www.cs.umass.edu/lfw/)
The image files "dog.jpg" & "2007_007763.jpg" are from dlib's github repository (https://github.com/davisking/dlib/blob/master/examples/faces)