LinkZyy

flashinfer

FlashInfer: Kernel Library for LLM Serving

vattention

Dynamic Memory Management for Serving LLMs without PagedAttention

DietCode

DietCode Code Release

cuda_ioctl_sniffer

Sniff CUDA ioctls

AutoAWQ

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

Retrieval_Head

open-source code for paper: Retrieval Head Mechanistically Explains Long-Context Factuality

academicpages.github.io

Github Pages template for academic personal websites, forked from mmistakes/minimal-mistakes

grok-1

Grok open release

FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

langchain

🦜🔗 Build context-aware reasoning applications

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

sglang

SGLang is yet another fast serving framework for large language models and vision language models.

souffle-ae

triton

Development repository for the Triton language and compiler

d2l-zh

《动手学深度学习》：面向中文读者、能运行、可讨论。中英文版被70多个国家的500多所大学用于教学。

StreamDiffusion

StreamDiffusion: A Pipeline-Level Solution for Real-Time Interactive Generation

SDL

Simple Directmedia Layer

in-context-ralm

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

punica

Serving multiple LoRA finetuned LLM as one

MemGPT

Create LLM agents with long-term memory and custom tools 📚🦙

LLMSpeculativeSampling

Fast inference from large lauguage models via speculative decoding

Awesome-LLM-System-Papers

S-LoRA

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

MQuAKE

[EMNLP 2023] MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions

ACL2023-Retrieval-LM.github.io

https://acl2023-retrieval-lm.github.io/

LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".

generative_agents

Generative Agents: Interactive Simulacra of Human Behavior