evaluation

There are 14 repositories under evaluation topic.

mrgloom / awesome-semantic-segmentation
:metal: awesome-semantic-segmentation
benchmark deeplearning evaluation semantic-segmentation
10357
langfuse
langfuse / langfuse
🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
analytics evals evaluation gpt langchain large-language-models llama-index llm llm-evaluation llmops monitoring observability open-source openai playground prompt-engineering prompt-management self-hosted ycombinator
Language:TypeScript 3851
Knetic / govaluate
Arbitrary expression evaluation for golang
evaluation expression go parsing
Language:Go 3575
sdiehl / write-you-a-haskell
Building a modern functional compiler from first principles. (http://dev.stephendiehl.com/fun/)
book compiler evaluation functional-language functional-programming haskel hindley-milner intermediate-representation lambda-calculus pdf-book type type-checking type-inference type-system type-theory
Language:Haskell 3319
evo
MichaelGrupp / evo
Python package for the evaluation of odometry and SLAM
benchmark euroc evaluation kitti mapping metrics odometry robotics ros ros2 slam trajectory trajectory-analysis trajectory-evaluation tum
Language:Python 3269
viebel / klipse
Klipse is a JavaScript plugin for embedding interactive code snippets in tech blogs.
clojure clojurescript interactive-snippets javascript code-evaluation ruby scheme prolog react reactjs klipse-plugin codemirror-editor evaluation python brainfuck lua ocaml reasonml common-lisp
Language:HTML 3100
promptfoo / promptfoo
Test your prompts, models, and RAGs. Catch regressions and improve prompt quality. LLM evals for OpenAI, Azure, Anthropic, Gemini, Mistral, Llama, Bedrock, Ollama, and other local & private models with CI/CD integration.
ci ci-cd cicd evaluation evaluation-framework llm llm-eval llm-evaluation llm-evaluation-framework llmops prompt-engineering prompt-testing prompts rag testing
Language:TypeScript 2986
zzw922cn / Automatic_Speech_Recognition
End-to-end Automatic Speech Recognition for Madarian and English in Tensorflow
automatic-speech-recognition tensorflow timit-dataset feature-vector phonemes data-preprocessing rnn audio deep-learning lstm end-to-end cnn rnn-encoder-decoder evaluation paper speech-recognition layer-normalization chinese-speech-recognition
Language:Python 2839
open-compass / opencompass
OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.
benchmark chatgpt evaluation large-language-model llama2 llama3 llm openai
Language:Python 2799
CLUEbenchmark / SuperCLUE
SuperCLUE: 中文通用大模型综合性基准 | A Benchmark for Foundation Models in Chinese
chatgpt chinese evaluation foundation-models gpt-4
2687
microsoft / promptbench
A unified evaluation framework for large language models
adversarial-attacks benchmark chatgpt evaluation large-language-models prompt prompt-engineering robustness
Language:Python 2117
ianarawjo / ChainForge
An open-source visual programming environment for battle-testing prompts to LLMs.
ai evaluation large-language-models llmops llms prompt-engineering
Language:TypeScript 2036
uptrain-ai / uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
autoevaluation evaluation experimentation hallucination-detection jailbreak-detection llm-eval llm-prompting llm-test llmops machine-learning monitoring openai-evals prompt-engineering root-cause-analysis
Language:Python 2033
huggingface / evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
evaluation machine-learning
Language:Python 1844
Cloud-CV / EvalAI
:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
ai ai-challenges angular7 angularjs artificial-intelligence challenge django docker evalai evaluation leaderboard machine-learning python reproducibility reproducible-research
Language:Python 1699
avalanche
ContinualAI / avalanche
Avalanche: an End-to-End Library for Continual Learning based on PyTorch.
benchmarks continual-learning continualai deep-learning evaluation framework library lifelong-learning metrics pytorch strategies training
Language:Python 1692
xinshuoweng / AB3DMOT
(IROS 2020, ECCVW 2020) Official Python Implementation for "3D Multi-Object Tracking: A Baseline and New Evaluation Metrics"
computer-vision machine-learning robotics tracking 3d-tracking multi-object-tracking real-time evaluation-metrics evaluation 3d-multi-object-tracking 2d-mot-evaluation kitti 3d-mot 3d-multi kitti-3d
Language:Python 1637
pycm
sepandhaghighi / pycm
Multi-class confusion matrix library in Python
accuracy ai artificial-intelligence classification confusion-matrix data data-analysis data-mining data-science deep-learning deeplearning evaluation machine-learning mathematics matrix ml multiclass-classification neural-network statistical-analysis statistics
Language:Python 1432
Maluuba / nlg-eval
Evaluation code for various unsupervised automated metrics for Natural Language Generation.
bleu bleu-score cider dialog dialogue evaluation machine-translation meteor natural-language-generation natural-language-processing nlg nlp rouge rouge-l skip-thought-vectors skip-thoughts task-oriented-dialogue
Language:Python 1304
MLGroupJLU / LLM-eval-survey
The official GitHub page for the survey paper "A Survey on Evaluation of Large Language Models".
benchmark evaluation large-language-models llm llms model-assessment
1264
abo-abo / lispy
Short and sweet LISP editing
navigation evaluation refactoring emacs-lisp clojure common-lisp scheme python
Language:Emacs Lisp 1182
tatsu-lab / alpaca_eval
An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
deep-learning evaluation foundation-models instruction-following large-language-models leaderboard nlp rlhf
Language:Jupyter Notebook 1174
EthicalML / xai
XAI - An eXplainability toolbox for machine learning
explainability xai ml ai bias artificial-intelligence bias-evaluation explainable-ai explainable-ml machine-learning machine-learning-explainability interpretability xai-library evaluation imbalance upsampling downsampling feature-importance
Language:Python 1066
fuzzbench
google / fuzzbench
FuzzBench - Fuzzer benchmarking as a service.
fuzzing benchmarking benchmark-framework evaluation security
Language:Python 1059
lunary-ai / lunary
The production toolkit for LLMs. Observability, prompt management and evaluations.
ai llm logs monitoring openai self-hosted langchain testing hacktoberfest evaluation observability prompts
Language:TypeScript 894
torch-fidelity
toshas / torch-fidelity
High-fidelity performance metrics for generative models in PyTorch
evaluation frechet-inception-distance gan generative-model inception-score kernel-inception-distance metrics perceptual-path-length precision pytorch reproducibility reproducible-research
Language:Python 890
PRBonn / semantic-kitti-api
SemanticKITTI API for visualizing dataset, processing data, and evaluating results.
dataset deep-learning evaluation labels large-scale-dataset machine-learning semantic-scene-completion semantic-segmentation
Language:Python 726
PaesslerAG / gval
Expression evaluation in golang
evaluate-expressions expression-evaluator gval expression-language parsing godoc evaluation golang parser go
Language:Go 702
dbolya / tide
A General Toolbox for Identifying Object Detection Errors
object-detection instance-segmentation evaluation toolbox errors error-detection
Language:Python 691
bochinski / iou-tracker
Python implementation of the IOU Tracker
tracker evaluation demo-script mot iou-tracker python detrac ua-detrac tracking-by-detection
Language:Python 688
CBLUEbenchmark / CBLUE
中文医疗信息处理基准CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark
benchmark chineseblue corpus dataset chinese biomedical-tasks evaluation acl2022
Language:Python 679
codingseb / ExpressionEvaluator
A Simple Math and Pseudo C# Expression Evaluator in One C# File. Can also execute small C# like scripts
calculations csharp-script eval evaluate evaluate-expressions evaluation evaluator execute executescript expression expression-evaluator expression-parser fluid math mathematical-expressions mathematical-expressions-evaluator parser reflection script scripting
Language:C# 564
autoprompt
ucinlp / autoprompt
AutoPrompt: Automatic Prompt Construction for Masked Language Models.
evaluation language-model nlp
Language:Python 558
tecnickcom / tcexam
TCExam is a CBA (Computer-Based Assessment) system (e-exam, CBT - Computer Based Testing) for universities, schools and companies, that enables educators and trainers to author, schedule, deliver, and report on surveys, quizzes, tests and exams.
cba cbt computer-based-assessment computer-based-testing e-exam tcexam exam school university testing evaluation essay multiple-choice mcsa mcma
Language:PHP 543
jkkummerfeld / text2sql-data
A collection of datasets that pair questions with SQL queries.
database dataset dynet evaluation natural-language-interface natural-language-processing neural-network nlp sql
Language:Python 508
zenogantner / MyMediaLite
recommender system library for the CLR (.NET)
collaborative-filtering evaluation item-prediction matrix-factorization rating-prediction recommender-systems
Language:C# 498