dpimagine

CMMLU

CMMLU: Measuring massive multitask language understanding in Chinese

mle-bench

MLE-bench is a benchmark for measuring how well AI agents perform at machine learning engineering

Hallu-PI

The code and datasets of our ACM MM 2024 paper "Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs".

Counting-Stars

Counting-Stars (★)

LLMTest_NeedleInAHaystack

Doing simple retrieval from LLM models at various context lengths to measure accuracy

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

data-juicer

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

Swin-Transformer

This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".

Image-Aesthetics-and-Quality-Assessment

[ACMMM 2023, Official Code] for paper "EAT: An Enhancer for Aesthetics-Oriented Transformers". Official Weights and Demos provided. 目前是地表最强开源美学评估模型之一.

ava_downloader

:arrow_double_down: Download AVA dataset (A Large-Scale Database for Aesthetic Visual Analysis)

Neural-IMage-Assessment

A PyTorch Implementation of Neural IMage Assessment

BAID

IAA_Tutorial

实验室【外部】美学课题组入门学习材料，加入课题组后，会有更详细的内部学习资料。

Image-Color-Aesthetics-and-Quality-Assessment

[ICCV 2023, Official Code] for paper "Thinking Image Color Aesthetics Assessment: Models, Datasets and Benchmarks". Official Weights and Demos provided. 首个面向图像色彩主观美学评估的数据集、算法和benchmark.

TANet-image-aesthetics-and-quality-assessment

[IJCAI 2022, Official Code] for paper "Rethinking Image Aesthetics Assessment: Models, Datasets and Benchmarks". Official Weights and Demos provided. 首个面向多主题场景的美学评估数据集、算法和benchmark.

GAOKAO-MM

[ACL'2024 Findings] GAOKAO-MM: A Chinese Human-Level Benchmark for Multimodal Models Evaluation

MathEval

MathEval is a benchmark dedicated to the holistic evaluation on mathematical capacities of LLMs.

Qwen2.5

Qwen2.5 is the large language model series developed by Qwen team, Alibaba Cloud.

MMBench

Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"

mercury

Convert Jupyter Notebooks to Web Apps

EffiBench

[NeurIPS 2024] EffiBench: Benchmarking the Efficiency of Automatically Generated Code

ianvs

Distributed Synergy AI Benchmarking

FlagGems

FlagGems is an operator library for large language models implemented in Triton Language.

MMTrustEval

A toolbox for benchmarking trustworthiness of multimodal large language models (MultiTrust, NeurIPS 2024 Track Datasets and Benchmarks)

amber-data-prep

Data preparation code for Amber 7B LLM

MobileLLM

MobileLLM Optimizing Sub-billion Parameter Language Models for On-Device Use Cases. In ICML 2024.

unibench

Python Library to evaluate VLM models' robustness across diverse benchmarks

MiniGPT-4-ZH

MiniGPT-4 中文部署翻译 完善部署细节

Qwen-VL

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

bookget

bookget 数字古籍图书下载工具