Frequency of Leaderboard Updates
brunnedu opened this issue · comments
Hey there,
We submitted to the leaderboard 2 days ago, and although the evaluation code ran successfully with the --submit flag, we are unsure if the leaderboard simply hasn't been updated yet or if there were any issues with our submission.
Also, we are wondering if the evaluation is re-run on your side or if the evaluation results from our local runs are used.
Hi @brunnedu, we got your submission and the leaderboard should be updated now. The evals are not re-run for now, but we may do so in the future.