OpenBioLink / ThoughtSource

A central, open resource for data and tools related to chain-of-thought reasoning in large language models. Developed @ Samwald research group: https://samwald.info/

OpenBioLink/ThoughtSource Issues

mawps dataset cot incorrect
Updated 7 months ago
New CoT Dataset Report: CoT-Collection
Updated 10 months ago
Tutorial notebook does not open
Closed 10 months ago1
Annotator sort by dataset
Closed 10 months ago
Add info on ThoughtSource-100 to Readme
Closed 10 months ago
Source mode not working for new datasets
Updated 10 months ago1
Generate takes very long time to finish
Closed a year ago3
krippendorff scores
Updated a year ago
from_json seems not to work for empty datasets
Closed a year ago2
Annotator "star" feature does not work properly
Updated a year ago
Add "_annotated" subscript to downloaded file from annotator
Updated a year ago
Annotator does not allow unselecting annotations
Updated a year ago
rename "cot" in config to "generated_cot"
Closed a year ago
Annotator does not show correct option for boolean dataset.
Closed a year ago2
save generated cots in case of error
Closed a year ago2
loading collections and generated_cots accept single strings, not only lists
Closed a year ago1
evaluation for duplicated answer choices
Updated a year ago1
link to Annotator example file broken
Closed a year ago4
throw error if from_json is used on already loaded collection
Updated a year ago
change saving of default template
Updated a year ago
PubmedQA dataset: Add reference CoT based on LONG_ANSWER field in source
Closed a year ago
add conda
Updated a year ago
Integrate context into template field possibilities
Updated a year ago
Add datasets to Hugging Face dataset hub
Updated a year ago
Check that contributor's guide is still up-to-date
Closed a year ago1
Default values for str if empty: None or ""
Closed a year ago2
Datasets: MedQA, MedMCQA, PubmedQA
Closed a year ago1
medmc_qa dataset does not contain generated CoTs
Closed a year ago1
Rename templates.json
Closed a year ago1
Document our standardized data schema
Closed a year ago2
Build tests sometimes fail because of "too many requests" HTTP error in our remote sources
Closed a year ago
`document_id` is redundant
Closed a year ago
Dataset: AQuA
Closed 2 years ago8
Grant application writing
Closed 2 years ago
Add datasets from 'Can large language models reason about medical questions?' paper (?)
Closed 2 years ago
Create script that generates an overview of our converted datasets and their contents
Closed 2 years ago1
Create first interactive demo for at least one task
Closed 2 years ago1
Create motivating demos of CoT streams
Closed 2 years ago1
Add markdown page describing added datasets (transfer from Google Doc)
Closed 2 years ago1
Update primary Readme page so it introduces project to broader public
Closed 2 years ago
Collect prompts used to generate CoTs
Closed 2 years ago1
Compile license info for all included datasets
Closed 2 years ago
Dataset: EntailmentBank
Closed 2 years ago17
Dataset: OpenBookQA
Closed 2 years ago1
Dataset: CommonsenseQA
Closed 2 years ago8
Reorganize Github repo to accomodate multiple subcomponents
Closed 2 years ago
Datasets: Datasets from Wei2022 repository (aqua, asdiv, commonsenseqa, date_understanding, gsm, mapwps sports_understanding, strategy_qa, svamp)
Closed 2 years ago7
Dataset: WorldTree
Closed 2 years ago5
Add field 'generated_answer' to schema
Closed 2 years ago
Error during pip install
Closed 2 years ago