alpaca_eval.jsonl

[Yunyi Ma]

Hi,

I am trying to follow your setup to reproduce the experimental results. While for the decoding stage, I do not find the eval .json file from the path you specified "data/alpaca_eval/alpaca_eval.jsonl". Would you mind to share the 'alpaca_eval.jsonl' file such that I can make the evaluation protocol consistent with your implementation?

Thanks!

[Tong Zhu (朱桐)]

Hi there, sorry for the confusion about the file name.

In case of anyone who's encountering the same problem, the alpaca_eval.jsonl can be downloaded from huggingface datasets, while the name alpaca_eval.json is changed into alpaca_eval.jsonl.

If you have any further questions or if you are willing to share your implementation experiences, please feel free to file an issue.