About evaluating on AFLW2000-3D
hao-qiang opened this issue · comments
hi,
You method need alignment the input according to 5 3d landmarks before inference, when evaluating on AFLW2000-3D, which landmarks detector do you use, or just use 5 landmarks from ground truth.
Thanks.
Using the landmark detector in our paper, the implement detail section.
- First, I detect 68 landmarks by face-alignment library, and bbox from gt (don't use sfd detector ).
img = cv2.imread('')[:,:,::-1]
fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D)
pts68_fa = fa.get_landmarks(img, detected_faces=[[letf, top, right, bottom]])[0]
- Then alignment by your code
_, img_warped, tform = crop_align_affine_transform(pts68_fa[:,:2], img, FLAGS.img_height, std_224_bfm09)
- Pred 68 lmks from MGCNet.
image_rgb_b = img_warped[None, ...]
pred = system.inference(sess, image_rgb_b)
lm_68 = pred['lm2d'][0][0,:,:]
-
Inverse affine transform pred lmks to source image.
-
Calculate NME by:
def calc_nme(pts68_pred, pts68_true):
minx, maxx = np.min(pts68_true[:, 0]), np.max(pts68_true[:, 0])
miny, maxy = np.min(pts68_true[:, 1]), np.max(pts68_true[:, 1])
llength = sqrt((maxx - minx) * (maxy - miny))
dis = pts68_pred - pts68_true
dis = np.sqrt(np.sum(np.power(dis, 2), 1))
dis = np.mean(dis)
nme = dis / llength
return nme
However the result is
[ 0, 30] Mean: 3.03
[30, 60] Mean: 3.83
[60, 90] Mean: 5.30
[ 0, 90] Mean: 4.05
if using face_alignment.LandmarksType._3D, the result is
[ 0, 30] Mean: 3.03
[30, 60] Mean: 3.85
[60, 90] Mean: 5.03
[ 0, 90] Mean: 3.97
if using gt lmks to alignment input rather than face_alignment, the result is
[ 0, 30] Mean: 2.87
[30, 60] Mean: 3.42
[60, 90] Mean: 3.98
[ 0, 90] Mean: 3.42
This result is closer to your paper.
And the image alignment by gt like the image in your paper.
image in your paper
alignment by gt
alignment by face-alignment
Is there any mistakes in my evaluation.
In my memory, we use face-alignment library.
I would like to check the code after November 17.
Could you check the code soon?Because we want to compare with your method in a fair way before the cvpr deadline. A lot of thanks.