🚀 Track with landmark 🚀
🚅 Track with synthesis 🚅
🧐 Landmarks (2D/3D) and bounding box 🧐
Lightning Track is a monocular face tracker built on FLAME. It provides optimized FLAME parameters and camera parameters, along with the bounding box and landmarks used during optimization.
Our tracker operates at a remarkable speed 🚀, processing 250 frames in approximately 60 seconds under landmark mode and 250 frames in around 200 seconds under synthesis mode.
Install step by step
conda create -n track python=3.9
conda install pytorch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 pytorch-cuda=11.8 -c pytorch -c nvidia
conda install -c fvcore -c iopath -c conda-forge fvcore iopath
conda install pytorch3d -c pytorch3d
pip3 install mediapipe tqdm rich lmdb einops colored ninja av opencv-python scikit-image onnxruntime-gpu onnx transformers pykalman
Install with environment.yml (recommend)
conda env create -f environment.yml
Run with Dockerfile
If your environment has unknown or unsolvable issues, use the Dockerfile as a final solution.
Check the build_resources.sh.
Track on video:
python track_video.py -v demos/demo.mp4 --synthesis
or track all videos in a directory:
python track_video.py -v demos/ --no_vis