Adding new datasets
lksenel opened this issue · comments
I am a Ph.D. student from Munich, currently working on prompting and I am a big fan of the resource that you created with promptsource project. I am currently working on the datasets in the raft benchmark (ought/raft as in the datasets), and I wanted to test my models with different prompts that I manually write. However, as far as I can see, these datasets are not currently included in promprsource, although being part of huggingface datasets, and it is not possible to add new datasets to the prompt source. Is this feature going to be possible in the future or can we make requests for some datasets to be added to the collection? Thanks in advance.
Kerem Senel
Hi @lksenel,
It's possible to load community datasets though it's not supported through the interface at the moment.
You would basically need to add "ought" to the set here:
promptsource/promptsource/templates.py
Line 28 in 9ea89d1
We have not seen a big demand in adding prompts for community datasets, but if it's the case, we can re-adjust the interface!