OpenCompass

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

VLMEvalKit

Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

T-Eval

[ACL2024] T-Eval: Evaluating Tool Utilization Capability of Large Language Models Step by Step

MMBench

Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"

BotChat

Evaluating LLMs' multi-round chatting capability via assessing conversations generated by two LLM instances.

GTA

[NeurIPS 2024 D&B Track] GTA: A Benchmark for General Tool Agents

CompassJudger

The All-in-one Judge Models introduced by Opencompass

DevEval

A Comprehensive Benchmark for Software Development.

GAOKAO-Eval

MathBench

[ACL 2024 Findings] MathBench: A Comprehensive Multi-Level Difficulty Mathematics Evaluation Dataset

MMBench-GUI

Official repo of "MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agents". It can be used to evaluate a GUI agent with a hierarchical manner across multiple platforms, including Windows, Linux, macOS, iOS, Android and Web.

ANAH

[ACL 2024] ANAH & [NeurIPS 2024] ANAH-v2 & [ICLR 2025] Mask-DPO

Ada-LEval

The official implementation of "Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks"

CompassVerifier

[EMNLP 2025] CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward

CriticEval

[NeurIPS 2024] A comprehensive benchmark for evaluating critique ability of LLMs

GPassK

[ACL 2025] Are Your LLMs Capable of Stable Reasoning?

ProSA

[EMNLP 2024 Findings] ProSA: Assessing and Understanding the Prompt Sensitivity of LLMs

Creation-MMBench

Assessing Context-Aware Creative Intelligence in MLLMs

CIBench

Official Repo of "CIBench: Evaluation of LLMs as Code Interpreter "

CompassBench

Demo data of CompassBench

SAGA

RaML

[Preprint 2025] Deciphering Trajectory-Aided LLM Reasoning: An Optimization Perspective

human-eval

Code for the paper "Evaluating Large Language Models Trained on Code"

CodeBench

lagent-cibench

ReasonZoo

.github

hinode

A clean documentation and blog theme for your Hugo site based on Bootstrap 5

oc_doc_website

storage