Evaluation metrics

[Yaya Sy]

Hi, thank you for sharing your work. Very interesting!

Which metrics do you use for zero-shot evaluation? lm-evaluation-harness proposes two metrics and I was wondering which one you use.

Thank you a lot.

[Yixin]

Hi! We report acc_norm for hellaswag, arc_challenge and openbookqa, and report acc for others.

[Yaya Sy]

Thank you very much!