eric11eca / curriculum-ling

Curriculum is a new format of NLI benchmark for evaluation of broad-coverage linguistic phenomena. This linguistic-phenomena-driven benchmark can serve as an effective tool for diagnosing model behavior and verifying model learning quality.

Home Page:https://aclanthology.org/2022.naacl-main.234

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Where can I find the benchmark?

BKHMSI opened this issue · comments

Hi,

Are there any updates as to when the benchmark will be released?

Thanks!

Hi,

Thanks for reaching out! Sorry, I have been super busy with my internship work, so only had some time recently to release the benchmark. You can either download it from this Github repository: all the datasets are under the benchmark/tasks folder. You can also use the benchmark/tasks_v1.0.0.zip. The benchmark is also available on HuggingFace Hub: https://huggingface.co/datasets/chenz16/curriculum_benchmark

Hope this helps.

No worries, thanks Zeming!