Realtime + WASM + blinking
pjebs opened this issue · comments
I was wondering if this library can be used for detecting blinks.
You have a real-time example using python. Since chrome now supports (experimental) shape detection and video camera access: https://paul.kinlan.me/face-detection/ & https://medium.com/@joomiguelcunha/lets-play-with-chrome-s-face-detection-api-ca13017a958f
I was wondering if using your WASM port it was able to detect if a face was blinking.
Yes, it's possible, but it would be more trickier than the Python implementation, because the cascade classifier used for facial landmark points detection is different than the one used on OpenCV for example in that sense, that it's using only two landmark points for one eye: one for the left side and one for the right side (no points marking the top and bottom of the eyelid). If I would used these extra two points it would be much easier to calculate with a simple Euclidean distance formula if the eye is open or closed. That's why in the Python example I used the hough transform method included into the OpenCV library.
Regarding the Shape API i'm afraid it supports only face detection, so it's not usable for us.
Is there anyway to get blink detection working on an electron app (main process), which is essentially node.js?
You might check for OpenCV bindings for Javascript. As I mentioned before in Pigo the only way to detect blinking is using some sort of Hough transformation algorithm.
With pupil detection and wasm (access to webcam), is it possible to detect blink rate by measuring the rate at which the pupils appear/disappear?
@pjebs unfortunately this is not possible with Pigo, since the pupil/eyes detection does not require the eyes lid to be opened.
Another possibility among the above mentioned Hough detection algorithm is to detect (calculate) the eye marker movement between a certain delta time, because you can observe that once you are closing the eye, the marker will move it's position rapidly. Here is a short video to make it more understandable.
I'll have to investigate your suggestion. Since blinks normally happen in under a few milliseconds, I suspect it won't be possible for normal webcams.
The duration of a blink is on average 100–150 milliseconds according