A unified benchmark for math reasoning.
original.zip contains the data used in the Lila paper.
We include it for reproducibility.
lila.zip contains the data for the official Lila benchmark,
which has undergone quality improvements since the paper.
You will need git-lfs to clone these,
or you can download them manually (e.g., lila.zip > download).
Contact matthewf@allenai.org with questions.
Do you have a math reasoning dataset that you would like to contribute? Open an issue, or email matthewf@allenai.org and we will work with you to incorporate it!
Cite this dataset and the source datasets (see sources.bib).
@INPROCEEDINGS{Mishra2022Lila,
author = {
Swaroop Mishra
and Matthew Finlayson
and Pan Lu
and Leonard Tang
and Sean Welleck
and Chitta Baral
and Tanmay Rajpurohit
and Oyvind Tafjord
and Ashish Sabharwal
and Peter Clark
and Ashwin Kalyan},
title = {Lila: A Unified Benchmark for Mathematical Reasoning},
booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
year = {2022}
}