evaluation-metrics

There are 2 repositories under evaluation-metrics topic.

confident-ai / deepeval
The LLM Evaluation Framework
evaluation-framework evaluation-metrics llm-evaluation llm-evaluation-framework llm-evaluation-metrics
Language:Python 1680
xinshuoweng / AB3DMOT
(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"
computer-vision machine-learning robotics tracking 3d-tracking multi-object-tracking real-time evaluation-metrics evaluation 3d-multi-object-tracking 2d-mot-evaluation kitti 3d-mot 3d-multi kitti-3d
Language:Python 1624
google-research / rliable
[NeurIPS'21 Outstanding Paper] Library for reliable evaluation on RL and ML benchmarks, even with only a handful of seeds.
benchmarking evaluation-metrics google machine-learning reinforcement-learning rl
Language:Jupyter Notebook 692
OCTIS
MIND-Lab / OCTIS
OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)
topic-modeling latent-dirichlet-allocation latent-semantic-analysis evaluation-metrics natural-language-processing non-negative-matrix-factorization neural-topic-models bayesian-optimization hyperparameter-optimization hyperparameter-tuning hyperparameter-search topic-models nlp nlproc nlp-library
Language:Python 681
jitsi / jiwer
Evaluate your speech-to-text system with similarity measures such as word error rate (WER)
wer automatic-speech-recognition python3 speech-to-text evaluation-metrics word-error-rate
Language:Python 528
up42 / image-similarity-measures
:chart_with_upwards_trend: Implementation of eight evaluation metrics to access the similarity between two images. The eight metrics are as follows: RMSE, PSNR, SSIM, ISSM, FSIM, SRE, SAM, and UIQ.
evaluation-metrics image machine-learning metrics p1
Language:Python 511
proycon / pynlpl
PyNLPl, pronounced as 'pineapple', is a Python library for Natural Language Processing. It contains various modules useful for common, and less common, NLP tasks. PyNLPl can be used for basic tasks such as the extraction of n-grams and frequency lists, and to build simple language model. There are also more complex data types and algorithms. Moreover, there are parsers for file formats common in NLP (e.g. FoLiA/Giza/Moses/ARPA/Timbl/CQL). There are also clients to interface with various NLP specific servers. PyNLPl most notably features a very extensive library for working with FoLiA XML (Format for Linguistic Annotation).
computational-linguistics evaluation-metrics folia language-modelling library linguistics machine-learning natural-language-processing nlp nlp-library python search-algorithms text-processing
Language:Python 475
COMET
Unbabel / COMET
A Neural Framework for MT Evaluation
artificial-intelligence evaluation-metrics machine-learning machine-translation natural-language-processing nlp
Language:Python 391
ranx
AmenRa / ranx
⚡️A Blazing-Fast Python Library for Ranking Evaluation, Comparison, and Fusion 🐍
comparison data-fusion evaluation evaluation-metrics information-retrieval information-retrieval-evaluation information-retrieval-metrics metasearch numba python rank-fusion ranking-metrics recommender-systems score-fusion
Language:Python 325
v-iashin / SpecVQGAN
Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)
audio audio-generation bmvc evaluation-metrics gan melgan multi-modal pytorch transformer vas vggsound video video-features video-understanding vqvae
Language:Jupyter Notebook 316
huggingface / lighteval
LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.
evaluation evaluation-framework evaluation-metrics huggingface
Language:Python 314
relari-ai / continuous-eval
Open-Source Evaluation for GenAI Application Pipelines
evaluation-framework evaluation-metrics information-retrieval llm-evaluation llmops rag retrieval-augmented-generation
Language:Python 297
AgentOps-AI / agentops-py
Python SDK for agent evals and observability
agent agentops ai evals evaluation-metrics llm observability
Language:Python 272
salesforce / factCC
Resources for the "Evaluating the Factual Consistency of Abstractive Text Summarization" paper
text-summarization evaluation-metrics
Language:Python 260
bheinzerling / pyrouge
A Python wrapper for the ROUGE summarization evaluation package
rouge summarization evaluation-metrics nlp
Language:Python 244
clovaai / generative-evaluation-prdc
Code base for the precision, recall, density, and coverage metrics for generative models. ICML 2020.
deep-learning generative-adversarial-network evaluation-metrics precision recall machine-learning generative-model fidelity diversity evaluation icml icml-2020 icml2020
Language:Python 234
davidsbatista / NER-Evaluation
An implementation of a full named-entity evaluation metrics based on SemEval'13 Task 9 - not at tag/token level but considering all the tokens that are part of the named-entity
named-entity-recognition evaluation-metrics notebook-jupyter crfsuite semeval-2013 ner-evaluation ner semeval
Language:Python 206
FuxiaoLiu / LRV-Instruction
[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning
evaluation gpt-4 hallucination object-detection vision vqa llama vicuna llava gpt multimodal prompt-engineering chatgpt evaluation-metrics foundation-models vision-and-language iclr iclr2024
Language:Python 205
sharmaroshan / Twitter-Sentiment-Analysis
It is a Natural Language Processing Problem where Sentiment Analysis is done by Classifying the Positive tweets from negative tweets by machine learning models for classification, text mining, text analysis, data analysis and data visualization
nlp sentiment-analysis data-analysis bag-of-words data-visualization eda machine-learning classification cross-validation evaluation-metrics count-vectorizer wordcloud hashtags datacleaning
Language:Jupyter Notebook 197
clovaai / CLEval
CLEval: Character-Level Evaluation for Text Detection and Recognition Tasks
end-to-end-ocr evaluation-metrics text-detection text-detection-recognition text-recognition
Language:Python 184
TonicAI / tonic_validate
Metrics to evaluate the quality of responses of your Retrieval Augmented Generation (RAG) applications.
evaluation-framework evaluation-metrics large-language-models llm llmops llms rag retrieval-augmented-generation
Language:Python 174
tagucci / pythonrouge
Python wrapper for evaluating summarization quality by ROUGE package
summarization rouge natural-language-processing document-summarization python evaluation-metrics text-summarization
Language:Perl 165
feralvam / easse
Easier Automatic Sentence Simplification Evaluation
text-simplification sentence-simplification evaluation-metrics natural-language-generation natural-language-processing nlproc
Language:Roff 153
lartpang / PySODEvalToolkit
PySODEvalToolkit: A Python-based Evaluation Toolbox for Salient Object Detection and Camouflaged Object Detection
python3 metrics metrics-visualization pr-curve fm-curve s-measure f-measure e-measure mae saliency saliency-detection salient-object-detection latex evaluation evaluation-metrics evaluation-framework evaluator camouflaged-object-detection co-saliency co-salient-object-detection
Language:Python 143
MantisAI / nervaluate
Full named-entity (i.e., not tag/token) evaluation metrics based on SemEval’13
evaluation-metrics machine-learning named-entity-recognition natural-language-processing sequence-models
Language:Python 138
fakufaku / fast_bss_eval
A fast implementation of bss_eval metrics for blind source separation
audio-processing blind-source-separation bss evaluation-metrics fast
Language:Python 126
athina-ai / athina-evals
Python SDK for running evaluations on LLM generated responses
evaluation evaluation-framework evaluation-metrics llm-eval llm-evaluation llm-evaluation-toolkit llm-ops llmops
Language:Python 123
om-ai-lab / VL-CheckList
Evaluating Vision & Language Pretraining Models with Objects, Attributes and Relations.
evaluation-metrics multimodal-deep-learning vision-and-language
Language:Python 119
YuanXinCherry / Person-reID-Evaluation
GOM：New Metric for Re-identification. 👉GOM explicitly balances the effect of performing retrieval and verification into a single unified metric.
re-identification evaluation-metrics
Language:Python 108
tohinz / semantic-object-accuracy-for-generative-text-to-image-synthesis
Code for "Semantic Object Accuracy for Generative Text-to-Image Synthesis" (TPAMI 2020)
image-generation image-synthesis ms-coco evaluation-metrics gan gan-evaluation
Language:Python 105
LAIT-CVLab / TopPR
NeurIPS 2023 - TopP&R: Robust Support Estimation Approach for Evaluating Fidelity and Diversity in Generative Models Official Code
evaluation-metrics generative-model topological-data-analysis
Language:Python 101
msmsajjadi / precision-recall-distributions
Assessing Generative Models via Precision and Recall (official repository)
adversarial-networks evaluation evaluation-metrics gan generative-adversarial-network generative-models vae variational-autoencoder
Language:Python 99
tanyuqian / ctc-gen-eval
EMNLP 2021 - CTC: A Unified Framework for Evaluating Natural Language Generation
dialogue-generation evaluation-metrics summarization text-generation style-transfer
Language:Python 94
Muhtasham / summarization-eval
📝 Reference-Free automatic summarization evaluation with potential hallucination detection
evaluation-metrics summarization
Language:Python 92
hpclab / rankeval
Official repository of RankEval: An Evaluation and Analysis Framework for Learning-to-Rank Solutions.
learning-to-rank evaluation-framework evaluation-metrics analysis-framework ensemble-models regression-trees
Language:Python 87
Coldmist-Lu / ErrorAnalysis_Prompt
:gift:[ChatGPT4MTevaluation] ErrorAnalysis Prompt for MT Evaluation in ChatGPT
chatgpt evaluation-metrics machine-translation natural-language-processing prompt-learning
Language:Python 82

evaluation-metrics

confident-ai / deepeval

xinshuoweng / AB3DMOT

google-research / rliable

MIND-Lab / OCTIS

jitsi / jiwer

up42 / image-similarity-measures

proycon / pynlpl

Unbabel / COMET

AmenRa / ranx

v-iashin / SpecVQGAN

huggingface / lighteval

relari-ai / continuous-eval

AgentOps-AI / agentops-py

salesforce / factCC

bheinzerling / pyrouge

clovaai / generative-evaluation-prdc

davidsbatista / NER-Evaluation

FuxiaoLiu / LRV-Instruction

sharmaroshan / Twitter-Sentiment-Analysis

clovaai / CLEval

TonicAI / tonic_validate

tagucci / pythonrouge

feralvam / easse

lartpang / PySODEvalToolkit

MantisAI / nervaluate

fakufaku / fast_bss_eval

athina-ai / athina-evals

om-ai-lab / VL-CheckList

YuanXinCherry / Person-reID-Evaluation

tohinz / semantic-object-accuracy-for-generative-text-to-image-synthesis

LAIT-CVLab / TopPR

msmsajjadi / precision-recall-distributions

tanyuqian / ctc-gen-eval

Muhtasham / summarization-eval

hpclab / rankeval

Coldmist-Lu / ErrorAnalysis_Prompt