vLLM

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

aibrix

Cost-efficient and pluggable Infrastructure components for GenAI inference

llm-compressor

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

semantic-router

Intelligent Router for Mixture-of-Models

production-stack

vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

guidellm

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

recipes

Common recipes to run vLLM

compressed-tensors

A safetensors extension to efficiently store sparse quantized tensors on disk

tpu-inference

TPU inference for vLLM, with unified JAX and PyTorch support.

flash-attention

Fast and memory-efficient exact attention

speculators

A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM

dashboard

vLLM performance dashboard

vllm-spyre

Community maintained hardware plugin for vLLM on Spyre

ci-infra

This repo hosts code for vLLM CI & Performance Benchmark infrastructure.

vllm-openvino

vllm-project.github.io

vllm-nccl

Manages vllm-nccl dependency

vllm-gaudi

Community maintained hardware plugin for vLLM on Intel Gaudi

vllm-neuron

Community maintained hardware plugin for vLLM on AWS Neuron

vllm-xpu-kernels

The vLLM XPU kernels for Intel GPU

vllm-project.github.io-static

FlashMLA

media-kit

vLLM Logo Assets

rfcs

DeepGEMM

DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling