relari-ai / continuous-eval

Data-Driven Evaluation for LLM-Powered Applications

https://continuous-eval.docs.relari.ai/

relari-ai/continuous-eval Issues

Metadata comparison for retrieval metrics
Updated 3 months ago1
LLMBasedFaithfulness does not work when using LLMs other than OpenAI
Updated 3 months ago2
LLMBasedFaithfulness fails when response cannot be split into score and reasoning
Updated 3 months ago
Integration with DSPy
Updated 4 months ago4
Add Custom Metric Creation
Closed 4 months ago1
Aggregate classification metrics
Closed 4 months ago6
Add CONTRIBUTING.md
Closed 4 months ago1
Langchain dependency switch
Closed 5 months ago2
Enhancement: Integration with LiteLLM
Updated 5 months ago4
Highlight LLM Based Custom Metric in ReadMe and Docs
Updated 6 months ago
Handling context limit
Updated 6 months ago
Doc fixes
Closed 6 months ago
Add support for Cohere LLM
Closed 6 months ago2
Change default matching strategy to RougeChunkMatch
Closed 7 months ago1
No support for Azure OpenAI endpoints
Closed 7 months ago2
Fix typos in generation LLM metric docstrings
Closed 7 months ago
Fix docs link on home page
Closed 7 months ago
Should only throw error for missing OpenAI key if needed
Closed 8 months ago1
Typo: Rename Blue to BLEU
Closed 8 months ago1
Error handling for generation LLM metrics
Updated 8 months ago
Add LLM-based Answer Relevance / Style Consistency
Closed 8 months ago
Add Softmax Output for DebertaAnswerScores batch_calculate method
Closed 8 months ago1
Add BERT variations for faithfulness
Updated 9 months ago
Batch processing for semantic metrics
Closed 9 months ago