DCDmllm / Cheetah

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

About evaluation code

Scipio1996 opened this issue · comments

Thank you for providing such excellent work! But there seem to be no evaluation codes for I4. Hence, I want to ask if you can release the evaluation code, and then we can evaluate other models on I4.

Thank you in advance!

Thank you so much for your kind words and appreciation for our work!

To address your concern, we've indeed provided the evaluation scripts. You can find them at:
https://github.com/DCDmllm/Cheetah/tree/main/I4%20Benchmark/scripts

Moreover, the evaluation code specifically for Cheetor is available at:
https://github.com/DCDmllm/Cheetah/blob/main/Cheetah/i4_cheetah_llama2.py

Please let us know if you need further assistance or clarification on this matter. We're here to help!

Thank you for your kind response, I have found them!