The annex to article "Facial Expression Recognition using Distance Importance Scores Between of Facial Landmarks"
We extracted 2278 Euclidean distances from 120 thousand observations obtained as a result of frame-by-frame extraction of face landmarks from the RAVDESS and CREMA-D video databases. We used distances as input to the ensemble classifiers Random Forest Classifier (RFC), Extra Trees Classifier (ETC) and Ada-Boost Classi-fier (ABC) to feature importance scores. As a result of averaging the obtained scores and setting the threshold for evaluations of 0.009, 0.001 and 0.002, we received 368, 259 and 104 pairs of landmarks that are presented in Tables 1, 2 and 3, respectively.
Table 1. 368 pairs of landmarks for calculating distances
#
Pair
#
Pair
#
Pair
#
Pair
#
Pair
1
0, 16
75
10, 48
149
21, 54
223
33, 62
297
48, 64
2
0, 17
76
10, 54
150
21, 58
224
34, 63
298
48, 65
3
1, 45
77
10, 60
151
21, 59
225
35, 48
299
48, 66
4
1, 54
78
11, 14
152
21, 60
226
35, 50
300
48, 67
5
2, 26
79
11, 15
153
21, 67
227
35, 52
301
49, 51
6
2, 48
80
11, 16
154
22, 27
228
35, 53
302
49, 54
7
4, 17
81
11, 17
155
22, 28
229
36, 48
303
49, 55
8
4, 25
82
11, 18
156
22, 30
230
36, 60
304
49, 61
9
5, 17
83
11, 21
157
22, 39
231
36, 66
305
49, 64
10
5, 18
84
11, 46
158
22, 40
232
37, 39
306
49, 65
11
5, 23
85
13, 35
159
22, 42
233
37, 40
307
50, 52
12
5, 27
86
14, 23
160
22, 43
234
37, 41
308
50, 54
13
6, 19
87
14, 54
161
22, 44
235
37, 48
309
50, 62
14
6, 23
88
15, 64
162
22, 45
236
37, 60
310
50, 63
15
6, 25
89
16, 25
163
22, 46
237
37, 64
311
50, 64
16
6, 56
90
16, 64
164
22, 52
238
38, 40
312
51, 52
17
6, 64
91
17, 21
165
22, 54
239
38, 41
313
51, 53
18
7, 18
92
17, 25
166
22, 55
240
38, 43
314
51, 60
19
7, 19
93
17, 26
167
22, 56
241
38, 45
315
51, 63
20
7, 20
94
17, 36
168
22, 58
242
38, 48
316
52, 58
21
7, 21
95
17, 48
169
22, 64
243
38, 60
317
52, 60
22
7, 25
96
18, 21
170
22, 66
244
39, 48
318
52, 61
23
7, 39
97
18, 22
171
23, 27
245
39, 49
319
53, 59
24
7, 48
98
18, 23
172
23, 35
246
39, 57
320
53, 60
25
7, 54
99
18, 25
173
23, 36
247
39, 60
321
53, 61
26
7, 57
100
18, 26
174
23, 39
248
39, 61
322
53, 66
27
7, 60
101
18, 37
175
23, 42
249
39, 62
323
53, 67
28
7, 64
102
18, 41
176
23, 43
250
40, 48
324
54, 55
29
7, 65
103
18, 48
177
23, 45
251
41, 48
325
54, 56
30
8, 19
104
19, 22
178
23, 46
252
41, 60
326
54, 57
31
8, 20
105
19, 23
179
23, 47
253
42, 45
327
54, 58
32
8, 21
106
19, 31
180
23, 48
254
42, 52
328
54, 59
33
8, 22
107
19, 32
181
23, 55
255
42, 53
329
54, 60
34
8, 23
108
19, 37
182
24, 27
256
42, 54
330
54, 61
35
8, 24
109
19, 38
183
24, 42
257
42, 57
331
54, 62
36
8, 27
110
19, 39
184
24, 45
258
42, 58
332
54, 63
37
8, 28
111
19, 40
185
24, 46
259
42, 64
333
54, 65
38
8, 29
112
19, 41
186
24, 47
260
43, 46
334
54, 66
39
8, 30
113
19, 48
187
24, 64
261
43, 47
335
54, 67
40
8, 41
114
19, 49
188
24, 66
262
43, 54
336
55, 56
41
8, 43
115
20, 22
189
25, 47
263
43, 55
337
55, 59
42
8, 48
116
20, 23
190
25, 54
264
43, 58
338
55, 60
43
8, 49
117
20, 27
191
26, 45
265
43, 64
339
55, 64
44
8, 54
118
20, 28
192
26, 54
266
43, 65
340
56, 58
45
8, 55
119
20, 30
193
26, 64
267
44, 46
341
56, 59
46
8, 59
120
20, 38
194
27, 48
268
44, 47
342
56, 60
47
8, 60
121
20, 39
195
27, 64
269
44, 54
343
56, 63
48
8, 62
122
20, 40
196
28, 53
270
44, 55
344
56, 64
49
8, 63
123
20, 41
197
29, 33
271
44, 56
345
56, 65
50
8, 64
124
20, 42
198
29, 51
272
44, 58
346
57, 60
51
8, 66
125
20, 44
199
29, 61
273
44, 64
347
57, 64
52
8, 67
126
20, 48
200
29, 62
274
44, 65
348
57, 65
53
9, 18
127
20, 49
201
29, 63
275
44, 66
349
57, 66
54
9, 20
128
20, 51
202
30, 33
276
45, 54
350
58, 59
55
9, 21
129
20, 52
203
30, 50
277
45, 55
351
58, 60
56
9, 22
130
20, 54
204
30, 51
278
45, 56
352
58, 63
57
9, 23
131
20, 64
205
30, 53
279
45, 58
353
58, 64
58
9, 24
132
21, 22
206
30, 61
280
45, 64
354
58, 67
59
9, 25
133
21, 23
207
30, 62
281
46, 54
355
59, 60
60
9, 38
134
21, 24
208
30, 63
282
46, 64
356
59, 64
61
9, 39
135
21, 25
209
31, 33
283
47, 54
357
60, 61
62
9, 40
136
21, 27
210
31, 34
284
47, 64
358
60, 62
63
9, 44
137
21, 28
211
31, 35
285
48, 50
359
60, 63
64
9, 48
138
21, 29
212
31, 53
286
48, 52
360
60, 64
65
9, 54
139
21, 30
213
31, 56
287
48, 53
361
60, 65
66
9, 60
140
21, 36
214
31, 57
288
48, 54
362
60, 66
67
9, 64
141
21, 37
215
31, 64
289
48, 55
363
60, 67
68
10, 15
142
21, 38
216
32, 34
290
48, 56
364
61, 64
69
10, 18
143
21, 39
217
32, 35
291
48, 57
365
62, 64
70
10, 21
144
21, 40
218
32, 50
292
48, 58
366
64, 65
71
10, 24
145
21, 41
219
32, 61
293
48, 59
367
64, 66
72
10, 25
146
21, 42
220
33, 34
294
48, 61
368
64, 67
73
10, 27
147
21, 46
221
33, 35
295
48, 62
74
10, 43
148
21, 48
222
33, 61
296
48, 63
Table 2. 259 pairs of landmarks for calculating distances
#
Pair
#
Pair
#
Pair
#
Pair
#
Pair
1
0, 16
53
15, 64
105
22, 55
157
38, 40
209
48, 67
2
0, 17
54
16, 64
106
22, 64
158
38, 41
210
49, 51
3
1, 45
55
17, 36
107
23, 27
159
38, 43
211
49, 54
4
1, 54
56
17, 48
108
23, 42
160
38, 45
212
49, 61
5
2, 26
57
18, 22
109
23, 43
161
38, 48
213
49, 64
6
4, 17
58
18, 23
110
23, 45
162
39, 48
214
49, 65
7
4, 25
59
18, 25
111
23, 46
163
39, 57
215
50, 52
8
5, 27
60
18, 37
112
23, 47
164
39, 60
216
50, 63
9
6, 19
61
19, 22
113
23, 48
165
39, 62
217
50, 64
10
6, 23
62
19, 23
114
23, 55
166
40, 48
218
51, 60
11
6, 56
63
19, 31
115
24, 27
167
41, 48
219
52, 60
12
6, 64
64
19, 32
116
24, 42
168
42, 45
220
52, 61
13
7, 18
65
19, 37
117
24, 45
169
42, 52
221
53, 59
14
7, 19
66
19, 38
118
24, 46
170
42, 54
222
53, 60
15
7, 20
67
19, 39
119
24, 47
171
42, 57
223
54, 55
16
7, 25
68
19, 40
120
24, 64
172
42, 58
224
54, 56
17
7, 39
69
19, 41
121
25, 47
173
42, 64
225
54, 57
18
7, 48
70
19, 48
122
25, 54
174
43, 46
226
54, 58
19
7, 54
71
20, 22
123
26, 45
175
43, 47
227
54, 59
20
7, 60
72
20, 23
124
26, 54
176
43, 54
228
54, 60
21
7, 64
73
20, 27
125
26, 64
177
43, 64
229
54, 63
22
8, 19
74
20, 28
126
27, 48
178
44, 46
230
54, 65
23
8, 20
75
20, 30
127
28, 53
179
44, 47
231
54, 66
24
8, 21
76
20, 38
128
29, 33
180
44, 54
232
54, 67
25
8, 22
77
20, 39
129
29, 51
181
44, 55
233
55, 56
26
8, 23
78
20, 40
130
29, 61
182
44, 56
234
55, 59
27
8, 24
79
20, 41
131
29, 62
183
44, 58
235
55, 60
28
8, 27
80
20, 42
132
30, 50
184
44, 64
236
55, 64
29
8, 28
81
20, 48
133
30, 51
185
44, 65
237
56, 58
30
8, 41
82
21, 22
134
30, 61
186
44, 66
238
56, 59
31
8, 48
83
21, 23
135
30, 62
187
45, 54
239
56, 60
32
8, 49
84
21, 25
136
30, 63
188
45, 58
240
56, 64
33
8, 54
85
21, 27
137
31, 33
189
45, 64
241
57, 60
34
8, 55
86
21, 28
138
31, 34
190
46, 54
242
57, 64
35
8, 59
87
21, 30
139
31, 35
191
46, 64
243
57, 66
36
8, 60
88
21, 36
140
32, 34
192
47, 54
244
58, 59
37
8, 63
89
21, 37
141
32, 35
193
47, 64
245
58, 60
38
8, 64
90
21, 38
142
33, 34
194
48, 50
246
58, 64
39
8, 67
91
21, 39
143
33, 35
195
48, 52
247
58, 67
40
9, 21
92
21, 40
144
33, 61
196
48, 53
248
59, 64
41
9, 23
93
21, 41
145
33, 62
197
48, 54
249
60, 62
42
9, 24
94
21, 46
146
34, 63
198
48, 55
250
60, 63
43
9, 25
95
21, 48
147
35, 50
199
48, 56
251
60, 64
44
9, 48
96
21, 60
148
35, 52
200
48, 57
252
60, 65
45
9, 54
97
22, 27
149
35, 53
201
48, 58
253
60, 66
46
9, 60
98
22, 28
150
36, 48
202
48, 59
254
60, 67
47
9, 64
99
22, 30
151
36, 60
203
48, 61
255
61, 64
48
10, 25
100
22, 39
152
37, 39
204
48, 62
256
62, 64
49
10, 48
101
22, 40
153
37, 40
205
48, 63
257
64, 65
50
10, 54
102
22, 42
154
37, 41
206
48, 64
258
64, 66
51
10, 60
103
22, 46
155
37, 48
207
48, 65
259
64, 67
52
14, 54
104
22, 54
156
37, 64
208
48, 66
Table 3. 104 pairs of landmarks for calculating distances
#
Pair
#
Pair
#
Pair
#
Pair
#
Pair
1
0, 16
22
9, 48
43
23, 42
64
43, 46
85
54, 57
2
7, 19
23
9, 54
44
23, 43
65
43, 47
86
54, 58
3
7, 20
24
9, 60
45
23, 47
66
43, 54
87
54, 60
4
7, 48
25
9, 64
46
24, 46
67
44, 46
88
54, 65
5
7, 54
26
10, 54
47
25, 54
68
44, 47
89
54, 66
6
7, 60
27
17, 36
48
29, 33
69
44, 54
90
54, 67
7
8, 19
28
18, 25
49
30, 51
70
44, 64
91
55, 60
8
8, 20
29
19, 39
50
30, 62
71
45, 54
92
56, 60
9
8, 21
30
19, 40
51
31, 33
72
45, 64
93
57, 64
10
8, 24
31
20, 22
52
31, 34
73
46, 54
94
58, 64
11
8, 27
32
20, 39
53
31, 35
74
48, 54
95
59, 64
12
8, 48
33
20, 40
54
32, 35
75
48, 57
96
60, 63
13
8, 54
34
21, 22
55
33, 62
76
48, 58
97
60, 64
14
8, 55
35
21, 23
56
36, 48
77
48, 64
98
60, 65
15
8, 59
36
21, 37
57
37, 40
78
48, 65
99
60, 66
16
8, 60
37
21, 38
58
37, 41
79
48, 66
100
60, 67
17
8, 64
38
21, 48
59
38, 40
80
49, 54
101
61, 64
18
9, 21
39
22, 27
60
38, 41
81
50, 63
102
64, 65
19
9, 23
40
22, 30
61
38, 43
82
53, 60
103
64, 66
21
9, 24
41
22, 42
62
40, 48
83
54, 55
104
64, 67
22
9, 25
42
22, 54
63
41, 48
84
54, 56
Landmark incidence in 368, 259 and 104 features are shown in Figures 1, 2 and 3, respectively. As one can see from the figures 1, 2 and 3, most important landmarks are 48, 54, 60, 64, which are the corners of the mouth, 7, 8 and 9 are landmarks the angle of the chin, from 19 to 23 are landmarks the eyebrows.
Fig. 1. Landmark incidence in 368 features
Fig. 2. Landmark incidence in 259 features
Fig. 3. Landmark incidence in 104 features
About
The annex to article "Facial Expression Recognition using Distance Importance Scores Between of Facial Landmarks"