EleutherAI / lm-evaluation-harness

A framework for few-shot evaluation of language models.

Home Page:https://www.eleuther.ai

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature Request] pre-built Docker image support

zsaladin opened this issue · comments

Is there any plan to support pre-built docker image on dockerhub or github package?

I think It will be very helpful and become more accessible to a lot of people.

We'd be glad to support this! It's a great idea.

Won't be currently prioritizing it so if anyone is able to contribute this we'd welcome contributions. Else, will put it on the todo list.

An official lm-evaluation-harness Docker image would be great. I just found that BigScience hosts the following Docker image: https://github.com/orgs/bigcode-project/packages/container/package/evaluation-harness