Number replaced WE
abojchevski opened this issue · comments
With WE replaced numbers nala_training_51
P:0.8162 R:0.7947 F:0.8053 0 exact
P:0.4323 R:0.3879 F:0.4089 1 exact
P:0.4364 R:0.3810 F:0.4068 2 exact
P:0.6718 R:0.6322 F:0.6514 TOTAL exact
P:0.9219 R:0.8998 F:0.9107 0 overlapping
P:0.9338 R:0.8328 F:0.8804 1 overlapping
P:0.8831 R:0.8000 F:0.8395 2 overlapping
P:0.9222 R:0.8657 F:0.8931 TOTAL overlapping
vs the baseline nala_training_51
P:0.8157 R:0.7924 F:0.8039 0 exact
P:0.4105 R:0.3645 F:0.3861 1 exact
P:0.4727 R:0.4127 F:0.4407 2 exact
P:0.6687 R:0.6264 F:0.6469 TOTAL exact
P:0.9267 R:0.9046 F:0.9155 0 overlapping
P:0.9380 R:0.8317 F:0.8816 1 overlapping
P:0.8800 R:0.7952 F:0.8354 2 overlapping
P:0.9262 R:0.8675 F:0.8959 TOTAL overlapping
OLD WE
fold 0
tp:143 fp:79 fn:49 fpo:33 fno:35 P:0.6441 R:0.7448 F:0.6908 0 exact
tp:30 fp:80 fn:65 fpo:56 fno:49 P:0.2727 R:0.3158 F:0.2927 1 exact
tp:11 fp:15 fn:20 fpo:11 fno:12 P:0.4231 R:0.3548 F:0.3860 2 exact
tp:184 fp:174 fn:134 fpo:100 fno:96 P:0.5140 R:0.5786 F:0.5444 TOTAL exact
tp:143 fp:79 fn:49 fpo:33 fno:35 P:0.8210 R:0.9378 F:0.8755 0 overlapping
tp:30 fp:80 fn:65 fpo:56 fno:49 P:0.8491 R:0.8940 F:0.8710 1 overlapping
tp:11 fp:15 fn:20 fpo:11 fno:12 P:0.8947 R:0.8095 F:0.8500 2 overlapping
tp:184 fp:174 fn:134 fpo:100 fno:96 P:0.8370 R:0.9091 F:0.8716 TOTAL overlapping
fold 1
tp:100 fp:52 fn:26 fpo:14 fno:14 P:0.6579 R:0.7937 F:0.7194 0 exact
tp:32 fp:106 fn:80 fpo:80 fno:65 P:0.2319 R:0.2857 F:0.2560 1 exact
tp:17 fp:21 fn:19 fpo:13 fno:13 P:0.4474 R:0.4722 F:0.4595 2 exact
tp:149 fp:179 fn:125 fpo:107 fno:92 P:0.4543 R:0.5438 F:0.4950 TOTAL exact
tp:100 fp:52 fn:26 fpo:14 fno:14 P:0.7711 R:0.9143 F:0.8366 0 overlapping
tp:32 fp:106 fn:80 fpo:80 fno:65 P:0.8719 R:0.9219 F:0.8962 1 overlapping
tp:17 fp:21 fn:19 fpo:13 fno:13 P:0.8431 R:0.8776 F:0.8600 2 overlapping
tp:149 fp:179 fn:125 fpo:107 fno:92 P:0.8286 R:0.9134 F:0.8689 TOTAL overlapping
fold 2
tp:124 fp:61 fn:39 fpo:21 fno:18 P:0.6703 R:0.7607 F:0.7126 0 exact
tp:24 fp:61 fn:78 fpo:47 fno:45 P:0.2824 R:0.2353 F:0.2567 1 exact
tp:7 fp:14 fn:17 fpo:10 fno:11 P:0.3333 R:0.2917 F:0.3111 2 exact
tp:155 fp:136 fn:134 fpo:78 fno:74 P:0.5326 R:0.5363 F:0.5345 TOTAL exact
tp:124 fp:61 fn:39 fpo:21 fno:18 P:0.8030 R:0.8859 F:0.8424 0 overlapping
tp:24 fp:61 fn:78 fpo:47 fno:45 P:0.8923 R:0.7785 F:0.8315 1 overlapping
tp:7 fp:14 fn:17 fpo:10 fno:11 P:0.8750 R:0.8235 F:0.8485 2 overlapping
tp:155 fp:136 fn:134 fpo:78 fno:74 P:0.8411 R:0.8365 F:0.8388 TOTAL overlapping
fold 3
tp:236 fp:80 fn:53 fpo:34 fno:34 P:0.7468 R:0.8166 F:0.7802 0 exact
tp:49 fp:95 fn:88 fpo:75 fno:64 P:0.3403 R:0.3577 F:0.3488 1 exact
tp:7 fp:18 fn:15 fpo:7 fno:8 P:0.2800 R:0.3182 F:0.2979 2 exact
tp:292 fp:193 fn:156 fpo:116 fno:106 P:0.6021 R:0.6518 F:0.6259 TOTAL exact
tp:236 fp:80 fn:53 fpo:34 fno:34 P:0.8686 R:0.9412 F:0.9034 0 overlapping
tp:49 fp:95 fn:88 fpo:75 fno:64 P:0.9038 R:0.8868 F:0.8952 1 overlapping
tp:7 fp:18 fn:15 fpo:7 fno:8 P:0.6667 R:0.7586 F:0.7097 2 overlapping
tp:292 fp:193 fn:156 fpo:116 fno:106 P:0.8697 R:0.9113 F:0.8900 TOTAL overlapping
fold 4
tp:185 fp:84 fn:58 fpo:34 fno:33 P:0.6877 R:0.7613 F:0.7227 0 exact
tp:57 fp:106 fn:92 fpo:88 fno:69 P:0.3497 R:0.3826 F:0.3654 1 exact
tp:19 fp:34 fn:31 fpo:22 fno:25 P:0.3585 R:0.3800 F:0.3689 2 exact
tp:261 fp:224 fn:181 fpo:144 fno:127 P:0.5381 R:0.5905 F:0.5631 TOTAL exact
tp:185 fp:84 fn:58 fpo:34 fno:33 P:0.8344 R:0.9097 F:0.8705 0 overlapping
tp:57 fp:106 fn:92 fpo:88 fno:69 P:0.9224 R:0.9030 F:0.9126 1 overlapping
tp:19 fp:34 fn:31 fpo:22 fno:25 P:0.8462 R:0.9167 F:0.8800 2 overlapping
tp:261 fp:224 fn:181 fpo:144 fno:127 P:0.8693 R:0.9078 F:0.8881 TOTAL overlapping
number replaced
fold 0
tp:143 fp:79 fn:48 fpo:34 fno:36 P:0.6441 R:0.7487 F:0.6925 0 exact
tp:28 fp:78 fn:67 fpo:56 fno:51 P:0.2642 R:0.2947 F:0.2786 1 exact
tp:14 fp:12 fn:17 fpo:8 fno:9 P:0.5385 R:0.4516 F:0.4912 2 exact
tp:185 fp:169 fn:132 fpo:98 fno:96 P:0.5226 R:0.5836 F:0.5514 TOTAL exact
tp:143 fp:79 fn:48 fpo:34 fno:36 P:0.8256 R:0.9467 F:0.8820 0 overlapping
tp:28 fp:78 fn:67 fpo:56 fno:51 P:0.8599 R:0.8940 F:0.8766 1 overlapping
tp:14 fp:12 fn:17 fpo:8 fno:9 P:0.8857 R:0.7949 F:0.8378 2 overlapping
tp:185 fp:169 fn:132 fpo:98 fno:96 P:0.8422 R:0.9133 F:0.8763 TOTAL overlapping
fold 1
tp:101 fp:46 fn:25 fpo:13 fno:14 P:0.6871 R:0.8016 F:0.7399 0 exact
tp:34 fp:103 fn:78 fpo:79 fno:65 P:0.2482 R:0.3036 F:0.2731 1 exact
tp:19 fp:25 fn:17 fpo:12 fno:12 P:0.4318 R:0.5278 F:0.4750 2 exact
tp:154 fp:174 fn:120 fpo:104 fno:91 P:0.4695 R:0.5620 F:0.5116 TOTAL exact
tp:101 fp:46 fn:25 fpo:13 fno:14 P:0.7950 R:0.9209 F:0.8533 0 overlapping
tp:34 fp:103 fn:78 fpo:79 fno:65 P:0.8812 R:0.9319 F:0.9059 1 overlapping
tp:19 fp:25 fn:17 fpo:12 fno:12 P:0.7679 R:0.8958 F:0.8269 2 overlapping
tp:154 fp:174 fn:120 fpo:104 fno:91 P:0.8329 R:0.9233 F:0.8758 TOTAL overlapping
fold 2
tp:124 fp:55 fn:39 fpo:20 fno:17 P:0.6927 R:0.7607 F:0.7251 0 exact
tp:25 fp:57 fn:77 fpo:45 fno:42 P:0.3049 R:0.2451 F:0.2717 1 exact
tp:7 fp:14 fn:17 fpo:9 fno:10 P:0.3333 R:0.2917 F:0.3111 2 exact
tp:156 fp:126 fn:133 fpo:74 fno:69 P:0.5532 R:0.5398 F:0.5464 TOTAL exact
tp:124 fp:55 fn:39 fpo:20 fno:17 P:0.8214 R:0.8798 F:0.8496 0 overlapping
tp:25 fp:57 fn:77 fpo:45 fno:42 P:0.9032 R:0.7619 F:0.8266 1 overlapping
tp:7 fp:14 fn:17 fpo:9 fno:10 P:0.8387 R:0.7879 F:0.8125 2 overlapping
tp:156 fp:126 fn:133 fpo:74 fno:69 P:0.8519 R:0.8237 F:0.8375 TOTAL overlapping
fold 3
tp:234 fp:85 fn:55 fpo:38 fno:37 P:0.7335 R:0.8097 F:0.7697 0 exact
tp:49 fp:97 fn:88 fpo:75 fno:64 P:0.3356 R:0.3577 F:0.3463 1 exact
tp:8 fp:17 fn:14 fpo:7 fno:8 P:0.3200 R:0.3636 F:0.3404 2 exact
tp:291 fp:199 fn:157 fpo:120 fno:109 P:0.5939 R:0.6496 F:0.6205 TOTAL exact
tp:234 fp:85 fn:55 fpo:38 fno:37 P:0.8680 R:0.9450 F:0.9048 0 overlapping
tp:49 fp:97 fn:88 fpo:75 fno:64 P:0.8952 R:0.8868 F:0.8910 1 overlapping
tp:8 fp:17 fn:14 fpo:7 fno:8 P:0.6970 R:0.7931 F:0.7419 2 overlapping
tp:291 fp:199 fn:157 fpo:120 fno:109 P:0.8681 R:0.9155 F:0.8912 TOTAL overlapping
fold 4
tp:187 fp:80 fn:56 fpo:32 fno:32 P:0.7004 R:0.7695 F:0.7333 0 exact
tp:56 fp:107 fn:93 fpo:92 fno:74 P:0.3436 R:0.3758 F:0.3590 1 exact
tp:21 fp:32 fn:29 fpo:20 fno:23 P:0.3962 R:0.4200 F:0.4078 2 exact
tp:264 fp:219 fn:178 fpo:144 fno:129 P:0.5466 R:0.5973 F:0.5708 TOTAL exact
tp:187 fp:80 fn:56 fpo:32 fno:32 P:0.8395 R:0.9127 F:0.8746 0 overlapping
tp:56 fp:107 fn:93 fpo:92 fno:74 P:0.9367 R:0.9212 F:0.9289 1 overlapping
tp:21 fp:32 fn:29 fpo:20 fno:23 P:0.8421 R:0.9143 F:0.8767 2 overlapping
tp:264 fp:219 fn:178 fpo:144 fno:129 P:0.8775 R:0.9164 F:0.8965 TOTAL overlapping
NO REPLACEMENT:
tp:788 fp:356 fn:225 fpo:136 fno:134 P:0.6888 R:0.7779 F:0.7306 0 exact
tp:192 fp:448 fn:403 fpo:346 fno:292 P:0.3000 R:0.3227 F:0.3109 1 exact
tp:61 fp:102 fn:102 fpo:63 fno:69 P:0.3742 R:0.3742 F:0.3742 2 exact
tp:1041 fp:906 fn:730 fpo:545 fno:495 P:0.5347 R:0.5878 F:0.5600 TOTAL exact
tp:788 fp:356 fn:225 fpo:136 fno:134 P:0.8279 R:0.9208 F:0.8719 0 overlapping
tp:192 fp:448 fn:403 fpo:346 fno:292 P:0.8906 R:0.8820 F:0.8863 1 overlapping
tp:61 fp:102 fn:102 fpo:63 fno:69 P:0.8319 R:0.8540 F:0.8428 2 overlapping
tp:1041 fp:906 fn:730 fpo:545 fno:495 P:0.8522 R:0.8985 F:0.8747 TOTAL overlapping
YES REPLACEMENT:
tp:789 fp:345 fn:223 fpo:137 fno:136 P:0.6958 R:0.7796 F:0.7353 0 exact
tp:192 fp:442 fn:403 fpo:347 fno:296 P:0.3028 R:0.3227 F:0.3124 1 exact
tp:69 fp:100 fn:94 fpo:56 fno:62 P:0.4083 R:0.4233 F:0.4157 2 exact
tp:1050 fp:887 fn:720 fpo:540 fno:494 P:0.5421 R:0.5932 F:0.5665 TOTAL exact
tp:789 fp:345 fn:223 fpo:137 fno:136 P:0.8362 R:0.9243 F:0.8780 0 overlapping
tp:192 fp:442 fn:403 fpo:347 fno:296 P:0.8978 R:0.8864 F:0.8921 1 overlapping
tp:69 fp:100 fn:94 fpo:56 fno:62 P:0.8095 R:0.8539 F:0.8311 2 overlapping
tp:1050 fp:887 fn:720 fpo:540 fno:494 P:0.8573 R:0.9022 F:0.8791 TOTAL overlapping
Both NL & ST classes improve. We choose with replacement