An amazing explanation metric (diagnostic report) for text generation evaluation
We list all required dependencies in requirements.txt. You can create a conda environment and install all dependencies through following commands:
conda create -n instructscore python=3.9
conda activate instructscore
pip install -r requirements.txt
There are two ways to use InstructScore.
We have uploaded our model to Huggingface, which can be found here. You can directly try InstructScore via several lines of code:
from InstructScore import InstructScore
refs = ["Normally the administration office downstairs would call me when there’s a delivery."]
outs = ["Usually when there is takeaway, the management office downstairs will call."]
scorer = InstructScore()
batch_explanations, batch_scores = scorer.score(refs, outs)
You can also download the checkpoint from this Google Drive link.