multi-annotation-parser
A dependency parser for multi-annotation data
实验结果
from truth:从正确答案训练
ignore:忽略不一致的句子
copy：分割成两个句子
(a)loss:将双人标注不一致的答案各赋予0.5的权重参与计算
(b)loss:将双人标注不一致的答案以或的关系（联合标签）参与计算
weighted (a):a 加权
weighted (b):b 加权
weighted (a) and label：a 加权，同时加权label
mv: weighted majarity voting
| 批次 | ctb | hit | pmt | pmt2020重标 | ||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| train | dev | test | train | dev | test | train | dev | test | train | dev | test | |||||||||||||
| UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | UAS | LAS | |
| from truth | 100% | 100% | 66.77% | 62.22% | 66.43% | 60.85% | 99.93% | 99.87% | 69.91% | 63.78% | 68.22% | 62.13% | 99.93% | 99.89% | 51.27% | 42.33% | 50.91% | 40.27% | 100% | 99.99% | 57.40% | 48.36% | 58.39% | 49.26% |
| ignore | 100% | 100% | 66.01% | 60.98% | 65.50% | 59.64% | 99.97% | 99.97% | 69.29% | 62.74% | 68.01% | 61.51% | 99.98% | 99.96% | 47.86% | 38.58% | 47.81% | 37.32% | 100% | 99.99% | 55.29% | 44.71% | 53.89% | 44.48% |
| copy | 87.81% | 85.02% | 67.44% | 62.66% | 66.51% | 61.13% | 88.46% | 84.93% | 68.95% | 62.39% | 67.96% | 61.71% | 80.91% | 75.09% | 50.96% | 41.12% | 49.51% | 38.83% | 80.45% | 74.40% | 53.67% | 41.58% | 53.07% | 41.89% |
| （a）loss | 89.57% | 87.22% | 67.07% | 61.70% | 66.13% | 60.89% | 88.69% | 71.44% | 69.07% | 52.66% | 68.87% | 52.36% | 79.53% | 72.96% | 51.02% | 40.89% | 50.49% | 39.61% | 73.84% | 66.27% | 53.31% | 41.14% | 52.81% | 41.29% |
| （b）loss | 90.85% | 88.60% | 65.33% | 60.13% | 65.59% | 60.40% | 91.20% | 71.79% | 69.87% | 53.03% | 68.89% | 51.91% | 81.91% | 75.37% | 50.96% | 41.00% | 50.63% | 39.57% | 74.62% | 65.96% | 52.41% | 41.62% | 54.50% | 43.13% |
| weighted（a） | 89.94% | 87.25% | 67.74% | 62.72% | 66.92% | 61.47% | 89.00% | 85.11% | 70.05% | 64.09% | 68.27% | 62.04% | 80.53% | 74.31% | 52.14% | 42.33% | 50.24% | 39.91% | 73.86% | 65.05% | 52.90% | 40.28% | 52.83% | 41.13% |
| weighted（b） | 91.22% | 88.52% | 67.15% | 62.22% | 66.78% | 61.34% | 91.27% | 88.05% | 69.83% | 63.58% | 68.78% | 62.53% | 82.20% | 75.68% | 51.04% | 40.78% | 49.77% | 39.39% | 74.87% | 66.46% | 53.35% | 41.46% | 54.44% | 43.72% |
| ewighted（b）and label | 91.22% | 89.21% | 67.82% | 62.70% | 66.63% | 60.98% | 91.32% | 88.54% | 70.01% | 63.88% | 68.44% | 62.18% | 80.98% | 74.41% | 50.68% | 40.38% | 49.91% | 39.21% | 74.67% | 66.59% | 51.85% | 40.57% | 52.37% | 41.47% |
| mv | 99.76% | 99.69% | 64.63% | 59.45% | 63.59% | 58.02% | 99.99% | 99.97% | 68.38% | 61.57% | 66.33% | 59.85% | 99.98% | 99.95% | 48.42% | 38.41% | 47.39% | 37.07% | 99.99% | 99.98% | 50.79% | 39.59% | 50.94% | 39.80% |