An implementation for Faithful Question Answering with Monte-Carlo Planning.
- Python 3.8
- Ubuntu 22.04
- Python Packages
conda create -n fame python=3.8
conda activate fame
pip install -r requirements.txt
Download Data folder.
The Data folder includes the EntailmentBank, EntailmentBankQA, training data for the controller, and training data for the verifier.
See Data/Readme.md
for details.
Download our trained models for direct reproduction, including
Controller,
Entailment Module,
Retriever,
and Verifier.
Unzip the files and place them in exp/
folder. Run the following commands to reproduce the results.
sh scripts/eval_scripts/reason_EBQA.sh
The result will be saved in the save_dir
.
sh scripts/eval_scripts/reason_EB.sh
The result will be saved in the save_dir
.
Use the offical evaluation code of EntailmentBank to evaluate automatically.
Please refer to MetGen for the training of the single-step entialment module.
sh scripts/training_scripts/train_Controller_ddp.sh
sh scripts/training_scripts/train_Retriever.sh
sh scripts/training_scripts/train_StepScorer.sh
@inproceedings{hong2023fame,
title={Faithful Question Answering with Monte-Carlo Planning},
author={Ruixin Hong, Hongming Zhang, Hong Zhao, Dong Yu and Changshui Zhang},
booktitle={The 61st Annual Meeting of the Association for Computational Linguistics ({ACL})},
year={2023}
}