Code was adapted from cleardusk's 3DDFA_V2 : https://github.com/cleardusk/3DDFA_V2
-
Module and web based API for ease of use.
-
3D facial landmarks via depth estimation and perspective reconstruction.
-
Supports inferencing on both ONNX runtime (CPU/GPU) as well as Pytorch.
-
Mesh interpolation and export with textured UV map in both
.obj
or EXPERIMENTAL.ply
. Note that the mesh is not unique enough to be used for identity verification since its shape is dependent of alignment. -
Capable of generating dlib landmarks (68 points), mesh landmarks (35568 points) or EXPERIMENTAL mediapipe/ARKit/Unity landmarks (468 points).
-
Clone this git repo by running
git clone https://github.com/JohnnyRacer/threedfa
-
Install requirements by running
pip install -r requirements.txt
-
Wait for the wheels to be built for your platform.
-
Run
python -m build
and then install the wheel usingpip install dist/threedfa.XXX.whl
- TODO : Building manylinux wheel for PyPi support.
Linux based distribution with gcc ⩾ 8.x or equivalent clang compiler for building cypthon versions of FaceBoxes and Sim3DR. Building on other OSes are untested.
torch torchvision pillow numpy opencv-python imageio imageio-ffmpeg pyyaml tqdm argparse cython scikit-image scipy onnxruntime
A relative new x86-64 CPU with decent floating point performance, with AVX support is preferable if using ONNX runtime on CPU. Most CPUs made after 2014 should give very satisfactory performance, with inference times of < 2 seconds for a 2K image. This code base has not been tested at all with ARM based CPUs, so compatibility and speeds are unknown. Please submit a PR if you are willing to share code or results related to ARM CPUs.
By default ONNX will be using the CPU for inference, GPU inference for ONNX can be enabled by uninstalling the onnxruntime
package from pip and replacing it with onnxruntime-gpu
. Please do note that the highest supported version of ONNX runtime is 1.8.0 for GPU inference. This is due to how ONNX has changed its providers allocation option parsing internally.
pip uninstall -y onnxruntime;
pip install onnxruntime-gpu==1.8.0
Untested. Although Pytorch XLA should be supported.
- Textured .obj files needs to implemented.
- Exporting to .ply is currently broken, I tried adapting the demo code from the source but had no luck in getting it to work.
- Partly obscured faces, particularly of the jaw will typically yield incorrectly aligned landmarks.
- Not a problem per se, but ONNX runtime on nvidia GPUs will require CUDA 11.0 and libCUDNN 8.0+. GPU Inference on AMD or Intel GPUs is not tested as I do not have access to such hardware. Please open an issue if you can verify and help test on these platforms.
Thanks to cleardusk for providing an amazing repo and codebase for 3DDFA V2
.
Thanks to rednafi for fastapi-nano
.