epfLLM / meditron

Meditron is a suite of open-source medical Large Language Models (LLMs).

Home Page:https://huggingface.co/epfl-llm

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Accuracy calculation failure

sia-cerebras opened this issue · comments

I see that here in evaluate.py accuracy is calculated in two different ways. And there is an assert statements to make sure they match.
In my tests, these assertion sometimes fails. Why is this assertion there? and what does it failure signify?

Thank you,