Giters

bigcode-project / bigcode-evaluation-harness

A framework for the evaluation of autoregressive code generation language models.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

run the MBPP in the HumanEval data format

virt9 opened this issue · comments

commented

Hello. thk you for your prior help! i want to run the MBPP dataset in the HumanEval data format. But I found that when i do this, the generation repeat until to the max_length ,but in HumanEval dataset it doesnt happen. so I wander if there is something I shoud change . perhaps the postprocess?