The locked-on is a simple implementation of the face locked-on video stabilization in python. Face lock-on is a type of stabilization, but in this case, we keep the face static.
A famous application of this type of technique was used on the Beats By Dre. The inspiration for this project came from the tutorial with after-effects on the Cinecom youtube channel. We can get much better results doing the process manually, but here we try to make automatic and free.
Disclaimer: Notice that this project was done with the purpose of study and practice. Need a set of optimization and tweaks.
Bellow, we can san an example, the input video contains a girl dancing on the left, and as output, we get just the video on her face fixed on the right.
We get each video and extract all the frames; on each frame, we detect the faces. With the faces detect, we set the center of the faces and calculate the variance between consecutive frames. With the face positions variance, we translate the images to the original center (the position the face on the first frame). Witht he translate image, we can get a black border, to avoid this, we crop the video.
The code was tested on python 3.6. The requirements can be installed by the requiriments.txt with pip install -r requiriments.txt. Bellow, we can see the requirements.
opencv-python==4.2.0.32face-recognition==1.3.0face-recognition-models==0.3.0numpy==1.18.1moviepy==1.0.2
The usage is pretty simple; you only need a video as input and run the script below.
python run_lock.py video_input_path video_output_path x_scale y_scale rate
The parameters are described bellow:
video_input_path: the video to be processedvideo_output_path: the output videox_scaleandy_scale: this parameter controls the border expands around the face detected if we want a bigger area around the face; we can tune this parameter.rate: since we run the detection on each frame, sometimes it can be time-consuming, this rate reduces the image size to detection. For instance, arate==2reduces the image input dimension by half. If the rate is to high, maybe the detection does not work.
- The video needs to be clean and with a good resolution.
- Work with one person at a time.
- It can take some time with large videos.
- Change the detections for object tracking.
I want to thank the face_recognition; this project mainly uses the tools provided by face_recognition.