It's crap but I spent more than 20 hours writing it.
It is a primitive face tracker, using OpenCV's built-in haarcascades classifier and minimum filtering.
Despite its name, it does not track shitty faces. It tracks beautiful faces, but the tracker itself is kinda bad. You could surely do better. I'm sharing it because there is not enough crap on the internet, and now I have better things to do.
I tried using state-of-the-art face detection algorithms and all of them have a documentation as pleasant as a big steaming pile of shit.
This Thing opens the camera, grabs frames, chews on them, and then outputs the frames to a v4l2 loopback device. Problem is, some video conferenceing stuff - jitsi, for example - won't list it because it isn't a capture device, but a video source. So this would still need some work by someone who isn't me.
Badly managed ADHD, appreciation for Ghost in the Shell, the Fediverse, and memes
Playing with machine vision can be fun, and can be also very frustrating. Ultimately it is a lot like sausage. Once you learn how it's made, you don't want it anymore.
You sure about that? Alright then.
Somehow get these things on your computer:
- Linux, or something that speaks V4L2 APIs reasonably well
- Video camera with a working driver
- Recent-ish OpenCV, Python 3, python-v4l2, python-opencv-python, python-imageio, v4l2loopback, python-argparse
I use arch btw. If you also do so, you can just
yay -S v4l2loopback-dkms python-imageio python-v4l2 python-argparse opencv python-opencv-python
You might need to tweak variables if it breaks. But it's just ./laughing-man.py
If you want to see a window with the processed video, use the --gui option.
If you don't want the v4l2loopback (why is it not called v4l3oopback?) use the --noloopback option.
Using both options makes the Shitty Face Tracker do absolutely nothing. I'm not sure if that makes it less useful.
To apply the Laughing Man overlay, you shoud activate it by echo 1 > /tmp/lm-ctrl
To deactivate, echo 0 > /tmp/lm-ctrl - I wanted to use SIGUSR1 and SIGUSR2 but that made OpenCV crash randomly...
This is implemented so that you can have some fun during video calls (if you manage to open the v4l3oopback in your particular video call thingy). But you can specify the --overlay_on option to have it default on.
Other options worth mentioning: --debug shows you frames around the tracked faces. Fading green (brightest: newest) are the valid tracked faces, fading red are the ones not qualifying as valid, and yellow are the recognized faces on the (future) frame. The image it is being drawn on is always one frame behind, so the tracker can see the future a little bit.
If the colors are mixed up on the loopback output, try --output_bgr. VLC and OBS apparently want different pixel formats...
Exit by pressing Ctrl-C over the terminal or q over the video window (when the --gui option was used).
Yes. Ideally not more than 2 frame latency, but YMMV.
Yes it does! You want to use the --output_bgr option in that case.
No problem, barely an inconvenience! That is actually why I started this whole thing. Just run ./face-tracker-cropper.py instead. It outputs to /dev/video2 what you can easily open in OBS.
Originally from Ghost in the Shell Stand Alone Complex, but this particular file is the work of Viscupelo from 2006. I used Glimpse to un-optimize the gif, crop it, and create an alpha mask.
I don't! OpenCV does with the haarcascade classifier.
Because alpha compositing is apparently not important enough to be part of OpenCV.
Mostly it does not!
The idea is to have a stateful face object for each face on the image. On each new frame I use haarcascades to detect faces (without apriori knowledge). The array of those faces (as rectangles) are then iterated over and I try to find close ones, update the face tracker with those positions, eliminate duplicates, and then add any new face which might have appeared.
My original idea was to use a sliding window of just 3 frames, and if on 2 or 3 of those frames a face was detected, it is considered valid. Otherwise it is discarded as noise.
Most certainly.
Yes but I can't bother.
In fact, I do. But it's difficult if not impossible to control what I find engaging in the moment.
Indeed! Follow me on fedi for more!
I do.
Alright, alright. I will finish this l