Head and eye gaze estimation
martinhoang11 opened this issue · comments
Hi, your project is very great !
I'm now working on head and eye estimation such as up-down, left-right directions problem. I've read your tracker.py file and I think you already have figured out this problem. Could you please guide which features or attributes I should use to solve this problem ?
Hi! For the head, the quaternion or euler and translation fields of the face object specify the rotation and position. You can calculate an estimate of the eye ball rotations by taking the rotation between the eyeball center point and the iris point as given here:
https://github.com/emilianavt/OpenSeeFace/blob/master/Unity/OpenSee.cs#L179-L180
The swapX calls are just for converting the coordinate system to Unity's and can be ignored.
About face detection, which frame work did you use for it, SSD, Yolo, Faster-RCNN for example ?
I just made something up myself. It's heatmap based, like the landmark detection, with an extra layer that gives the radius of bounding boxes and some max pooling to figure out local maximums of the heat map.
Understood, thank you very much !
Hi, I'm looking for codes that draw facial landmark points like cv2.circle(...), but I can't find out. Could you show me how did you draw these points ?
When I wrote that code, I didn't know about cv2.circle, so it's still setting the pixels manually, like here:
https://github.com/emilianavt/OpenSeeFace/blob/master/facetracker.py#L311-L346
I've been meaning to refactor it but haven't gotten around to it yet.