LMSYS

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

RouteLLM

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

arena-hard-auto

Arena-Hard-Auto: An automatic LLM benchmark.

llm-decontaminator

Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"

lm-sys.github.io

vicuna-blog-eval

The code and data for the GPT-4 based benchmark in the vicuna blog post