About evaluating on AFLW2000-3D

[haoqiang]

hi,
You method need alignment the input according to 5 3d landmarks before inference, when evaluating on AFLW2000-3D, which landmarks detector do you use, or just use 5 landmarks from ground truth.
Thanks.

[jiaxiangshang]

Using the landmark detector in our paper, the implement detail section.

[haoqiang]

1. First, I detect 68 landmarks by face-alignment library, and bbox from gt (don't use sfd detector ).

img = cv2.imread('')[:,:,::-1]
fa = face_alignment.FaceAlignment(face_alignment.LandmarksType._2D)
pts68_fa = fa.get_landmarks(img, detected_faces=[[letf, top, right, bottom]])[0]

2. Then alignment by your code

_, img_warped, tform = crop_align_affine_transform(pts68_fa[:,:2], img, FLAGS.img_height, std_224_bfm09)

3. Pred 68 lmks from MGCNet.

image_rgb_b = img_warped[None, ...]
pred = system.inference(sess, image_rgb_b)
lm_68 = pred['lm2d'][0][0,:,:]

4. Inverse affine transform pred lmks to source image.

5. Calculate NME by:

def calc_nme(pts68_pred, pts68_true):
    minx, maxx = np.min(pts68_true[:, 0]), np.max(pts68_true[:, 0])
    miny, maxy = np.min(pts68_true[:, 1]), np.max(pts68_true[:, 1])
    llength = sqrt((maxx - minx) * (maxy - miny))
    
    dis = pts68_pred - pts68_true
    dis = np.sqrt(np.sum(np.power(dis, 2), 1))
    dis = np.mean(dis)
    nme = dis / llength
    return nme

However the result is
[ 0, 30] Mean: 3.03
[30, 60] Mean: 3.83
[60, 90] Mean: 5.30
[ 0, 90] Mean: 4.05

if using face_alignment.LandmarksType._3D, the result is
[ 0, 30] Mean: 3.03
[30, 60] Mean: 3.85
[60, 90] Mean: 5.03
[ 0, 90] Mean: 3.97

if using gt lmks to alignment input rather than face_alignment, the result is
[ 0, 30] Mean: 2.87
[30, 60] Mean: 3.42
[60, 90] Mean: 3.98
[ 0, 90] Mean: 3.42
This result is closer to your paper.

And the image alignment by gt like the image in your paper.
image in your paper

alignment by gt

alignment by face-alignment

Is there any mistakes in my evaluation.

[jiaxiangshang]

In my memory, we use face-alignment library.
I would like to check the code after November 17.

[haoqiang]

Could you check the code soon？Because we want to compare with your method in a fair way before the cvpr deadline. A lot of thanks.