allanj / Deductive-MWP

Deductive Reasoning for Math Word Problem Solving

This is the offcial repo for the ACL-2022 paper "Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction".

Requirements

• transformers pip3 install transformers
• Pytorch > 1.7.1
• accelerate package pip3 install accelerate (for distributed training)

Usage

Reproduce the results, simply run our scripts under scripts folder.

Math23k

For example, reproduce the results for Math23k dataset with train/val/test setting,

bash scripts/run_math23k.sh

Run the following for the train/test setting

bash scripts/run_math23k_train_test.sh

Main Results

We reproduce the main results of Roberta-base-DeductiveReasoner in the following table.

Dataset	Value Accuracy
Math23k (train/val/test)	84.3
Math23k (train/test)	86.0
MAWPS (5-fold CV)	92.0
MathQA (train/val/test)	78.6
SVAMP	48.9

More details can be found in Appendix C in our paper.

Checkpoints

We also provide the Roberta-base-DeductiveReasoner checkpoints that we have trained on the Math23k, MathQA and SVAMP datasets. We do not provide the 5-fold model checkpoints due to space limitation.

Dataset	Link
Math23k (train/dev/test setting)	Link
Math23k (train/test setting)	Link
MathQA	Link
SVAMP	Link

Datasets

The data for Math23k Five-fold is not uploaded to GitHub due to slightly larger dataset size, it is uploaded here in Google Drive.

Citation

If you find this work useful, please cite our paper:

@inproceedings{jie2022learning,
  title={Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction},
  author={Jie, Zhanming and Li, Jierui and Lu, Wei},
  booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  pages={5944--5955},
  year={2022}
}