EleutherAI / lm-evaluation-harness

A framework for few-shot evaluation of language models.

Home Page:https://www.eleuther.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

No inference time is returned in results

giorgossideris opened this issue · comments

I think that it would be useful to return the inference time needed for the requests in the result.

Do you think that such a feature would be useful? If yes I can open a PR.

Just connecting other issue (not sure if it is the only one) about inference time report: #1236

Good, since there is demand for this feature I will open a PR.