Comparison to other AMR Coref Systems
bjascob opened this issue · comments
The only other system that I'm aware of is https://github.com/Sean-Blank/AMRcoref as described in the paper "End-to-End AMR Coreference Resolution". The scores in the paper are relatively similar to this project's scores.
There is also... https://github.com/IreneZihuiLi/VG-AMRCoref which has noticeably better scores.
The following is a quick comparison of the 3 different projects available on GitHub (including this one)
AMR Coref Comparisons 2022-07-02.odt
Re Fu et al. and Li et al work: the scores you mention are the scores they report in their work. However, I have tried to reproduce their work and failed. I contacted both authors, and while I had more exchange with Li, they both stopped responding to my correspondence.
This is very unfortunate given Fu et al. reports a F1 ave from 34 to 62.4 (28.4 improvement over the baseline) and Li et al. reports a F1ave from 62.4 to 73.5 (11.1 improvement over the baseline), which is now the new baseline.
For all practical purposes, amr_coref is the SOTA for others using AMR coref in their own research.