hendrycks / apps

APPS: Automated Programming Progress Standard (NeurIPS 2021)

hendrycks/apps Issues

Problem in ground-truth solutions
Updated a month ago
About Solutiions' validity
Closed a year ago1
check5 in function "run_test" seem to bring some wrong result
Updated a year ago2
Can this dataset test for chatgpt?(gpt 3.5?)
Closed a year ago10
Steps About Generated Code Solutions Post-processing
Updated a year ago1
Unable to run pre-trained (1.5B) model on test set
Updated a year ago1
Too Long Problems
Updated a year ago8
answer_type calculation is different for train/val and eval
Updated a year ago1
Problems with fine-tuning
Closed 2 years ago
Problems With APPS
Closed 2 years ago4
DeepSpeed config and TrainingArguments mismatch
Closed 2 years ago2
Request for scripts of fine-tuning
Closed 2 years ago3
Running instructions
Closed 2 years ago4
evaluation on multiple solutions at once causes memory leak
Closed 2 years ago14
Test case average of solutions in real dataset
Closed 2 years ago8
Nan test case average
Closed 2 years ago5
Computation of the accuracy scores when there are compilation and runtime errors
Closed 2 years ago7
Show a data instance in the readme
Closed 2 years ago2
Asking for scripts for pre-processing
Closed 2 years ago2
Request for pretrained models
Closed 2 years ago2
GPT-2 0.1B weights
Closed 3 years ago1
Categorization of Problem Difficulty
Closed 3 years ago3
Broken dataset download link
Closed 3 years ago2
Missing apps-train-files json file?
Closed 3 years ago5
Why don't add sorted(problems) to pin problems sorting?
Closed 3 years ago1
Wrong arXiv link in main read me file
Closed 3 years ago1