huggingface / evaluate

🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

Home Page:https://huggingface.co/docs/evaluate

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Adding new metrics locally without using the evaluate-cli command

sameeravithana opened this issue · comments

Is there any documentation to follow for adding new metrics without the evaluate-cli command and without pushing to HF spaces, but to work locally. For example, I develop a custom metric X, and create the metrics/X folder with the X.py and app.py. But I can not load it with the evaluate.load("X"). I think I am missing one step such that evaluate repository may not be able to find the correct module script for the metric. I installed the evaluate package in the editable mode using pip install -e .[dev]