git submodule update --init
-> initialize submodule SPIGA
This is a final project for CS445:Computational Photography at UIUC. Given input images of faces, automatic morphing is performed from one face to another.
![]() |
![]() |
![]() |
- Given two images (I1 and I2), facial keypoints are automatically identified and detected.
- Find average of corresponding points and get triangular mesh using Delaunay Triangulation
- To create frame t, interpolate corresponding points
- For each triangle, affine project to corresponding triangles in I1 and I2.
- Use affine projection to map pixel values from I1 and I2 to frame.
- Set final pixel value to be weighted average of pixel in I1 and I2.
- Frame t is done! Repeat for frame t+1
We implemented our own facial keypoint detector using training data from kaggle. This works well for in-distribution faces, but has difficulties in scaling to all faces due to training data size limitations. We also provide the option to leverage existing SOTA in facial keypoint detection SPIGA for better in-the-wild performance.
conda env create --name morph --file=environment.yml
custom_vis.ipynb -> Visualize custom trained face keypoint ResNet18 model detector_vis.ipynb -> visualize SPIGA face keypoint detection model image_morph_vis.ipynb -> visualize image morph with user input keypoint selection feature keypoint_detector_train.ipynb -> Training notebook for face keypoint detection
python3 engine.py $custom
-> takes in directory of images and morphs them into a video sequentially. Set argument $custom as 1 or 0 to use custom or SPIGA respectively.
https://www.kaggle.com/datasets/prashantarorat/facial-key-point-data
For training -> https://github.com/nalbert9/Facial-Keypoint-Detection