NeuralVFX / facial-pose-estimation-pytorch

Facial Pose Estimation using Pytorch

Facial-Pose-Estimation-Pytorch

Training a Neural Net in Pytorch to generate 3d facial pose estimation from only facial landmarks.

About

This is one of three repositories which together form a larger project, these are the three repositories:

• facial-pose-estimation-pytorch - You are here.
• facial-pose-estimation-opencv
• facial-pose-estimation-unity

This blog post describes the whole project: AR Facial Pose Estimation

Extra Info

• Main code is in Jupyter Notebook
• Training data is artificially generated on the fly using OpenCV
• Output is 51 blend-shape float values which can be applied to a face mesh
• This only outputs blend-shape values, head transform is handled by a separate part of the pipeline

Training Pipeline Example

Inference Pipeline Example

Artificial Data-Pair Examples

Code Usage

Usage instructions found here: user manual page.