OpenCompass

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

MixtralKit

A toolkit for inference and evaluation of 'mixtral-8x7b-32kseqlen' from Mistral AI

VLMEvalKit

Open-source evaluation toolkit of large vision-language models (LVLMs), support GPT-4v, Gemini, QwenVLPlus, 50+ HF models, 20+ benchmarks

LawBench

Benchmarking Legal Knowledge of Large Language Models

T-Eval

[ACL2024] T-Eval: Evaluating Tool Utilization Capability of Large Language Models Step by Step

BotChat

Evaluating LLMs' multi-round chatting capability via assessing conversations generated by two LLM instances.

MMBench

Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"

DevBench

A Comprehensive Benchmark for Software Development.

MathBench

[ACL 2024 Findings] MathBench: A Comprehensive Multi-Level Difficulty Mathematics Evaluation Dataset

Ada-LEval

The official implementation of "Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks"

CriticBench

A comprehensive benchmark for evaluating critique ability of LLMs

code-evaluator

A multi-language code evaluation tool.

OpenFinData

CodeBench

human-eval

Code for the paper "Evaluating Large Language Models Trained on Code"

.github

evalplus

EvalPlus for rigourous evaluation of LLM-synthesized code

pytorch_sphinx_theme

Sphinx Theme for OpenCompass - Modified from PyTorch