evaluate 时不考虑 token entity 正确性

[theblackcat102]

你好，想请问一下我发现你的 NER 在计算 F1 的时候似乎只有考虑 token 的位置正确性（ line 332） 但没有判断 entity 的分类正确性。我的理解是正确吗？

for entity in pred_entities_pos: if entity not in gold_entities_pos: continue else: correct += 1

[ahaliu1]

I have the same question. There are two different evaluation method of NER. One is strict-F1 which consider both positon and token. the other one is soft-F1 which consider positon or token. From the token perspective, It will be considered correct if the token is correct and predicated position has overlap with correct position. From the position perspective, It only consider the position.

So the author choose the soft-F1 measure and only consider position. It is the simplest measurement but it is not often used. I wonder the performance of the model under the strict-F1 measurement.