prometheus-eval / prometheus-eval

Evaluate your LLM's response with Prometheus and GPT4 💯

https://prometheus-eval.github.io/prometheus-eval/

prometheus-eval/prometheus-eval Issues

What are the rubricks for evaluation?
Updated 3 days ago1
Use of customized score/eval rubrics
Updated 5 days ago5
[Question] Support for including context (RAG)?
Closed 19 days ago3
Binary evaluation?
Closed 10 days ago3
Which directory is the relevant code for weight merging located in?
Closed 10 days ago1
small bug in relative_grade judge
Closed 10 days ago1
Can't load the model on T4 Inference GPU
Closed 19 days ago2
Feature Request: Release quantized version of Prometheus models
Updated 13 days ago3
Use prometheus LLM or other os LLMs through API like Ollama
Closed 18 days ago
feature request: use model from HuggingFace Inference endpoint
Closed 18 days ago1
Question about RuntimeError
Closed 19 days ago2
Best way to reuse the same model for direct assessment and pairwise ranking
Closed 19 days ago1
Question about multi-turn evaluation
Closed 2 months ago3
Speed-up strategies
Closed 2 months ago4
How to evaluate HHH, MT_Bench_human? Where to get human scores for other val sets?
Closed 2 months ago11
Reference Free metrics
Closed 2 months ago1
About the evaluation on direct assessment benchmarks
Closed 2 months ago3
Issue with examples
Closed 2 months ago4
llama3 evaluator
Closed 2 months ago2