cansik / auto-keypoint-retopology

Auto keypoint extractor for retopology.

Auto Keypoint Extractor for Retopology

The idea of this tool is to extract the facial keypoints of a texture, mark the vertices and export the information for retopology tools like R3DS Wrap.

General Concept

This is the general concept of the script. The actual implementation is more complex and for example includes multiple render steps.

1. Render camera image
2. Detect facial landmark keypoints
3. Project all vertices to camera space
4. Find vertices that match detected keypoints (KDTree)
5. Read uv coordinates of selected vertices
6. Export index and uv coordinates as JSON

Installation

Windows

Install Visual Studio Build Tools and make sure cmake.exe is in the path variable.

python3 -m ensurepip
python3 -m pip install --upgrade pip --user
python3 -m pip install cmake
python3 -m pip install opencv-python opencv-contrib-python imutils numpy scipy dlib --user

MacOS

./python3.7m -m ensurepip
./python3.7m -m pip install --upgrade pip --user
./python3.7m -m pip install opencv-python opencv-contrib-python imutils numpy scipy dlib --user

# dlib from source if needed
python3.7m setup.py install

Landmark Model

You can download it here http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2 or here.

Development

To have autocompletion in various IDE's for blender, install the following package:

pip3 install fake-bpy-module-2.80