oplatek / reprohum-as-little-as-possible

Description of reproduction effort for Section 5.2. of https://aclanthology.org/2022.acl-short.53/

With a Little Help from the Authors: Reproducing Human Evaluation of an MT Error Detector

Ondřej Plátek, Mateusz Lango and Ondřej Dušek

Our paper (see Arxiv) was accepted to HumEval’23 workshop.

We reproduced the human evaluation in the paper As little as possible, as much as necessary: Detecting over- and undertranslations with contrastive conditioning, see the Section 5.2.

Organization of this repository

• ./Common-Approach-to-Reproduction.pdf -- instructions from the ReproHum organizers for following the introductions.
• ./README.md This readme contains high-level info.
• The latex files for compiling the paper: acl_natbib.bst, ranlp2023.bib, ranlp2023.sty, ranlp2023.tex and the images/ folder for figures.
• heds-form-2022_11_18/ see the paper for how we used it. See the form for individual values.
• preproductions_instructions_as_recieved - At the beginning, the ReproHum organizers send us the information with instructions on how to reproduce the paper.

Acknowledgements

This work was co-funded by Charles University projects GAUK 40222, SVV 260575 and the European Union (ERC, NG-NLG, 101039303).