KevinZeng08

awsome-diffusion-serving

piperag

PipeRAG: Fast Retrieval-Augmented Generation via Algorithm-System Co-design

xDiT

A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters

mem0

The memory layer for Personalized AI

llm-action

本项目旨在分享大模型相关技术原理以及实战经验。

flashinfer

FlashInfer: Kernel Library for LLM Serving

MSVBASE

MSVBASE is a system that efficiently supports complex queries of both approximate similarity search and relational operators. It integrates high-dimensional vector indices into PostgreSQL, a relational database to facilitate complex approximate similarity queries.

radient

Radient turns many data types (not just text) into vectors for similarity search, RAG, regression analysis, and more.

GraphRAG_Lite

awesome-generative-information-retrieval

FastV

[ECCV 2024] Code for paper: An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models

curator

cuCollections

PGRAG

PGRAG

DB-GPT

AI Native Data App Development framework with AWEL(Agentic Workflow Expression Language) and Agents

MInference

To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.

ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

prompt-lookup-decoding

ESPN-v1

ESPN: Embedding from Storage Pipelined Network. GDS implementation for multi-vector embedding retrieval and bindings.

LOOK-M

Official implementation of "LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference"

LLM-Viewer

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

flux

A fast communication-overlapping library for tensor parallelism on GPUs.

Efficient-Multimodal-LLMs-Survey

Efficient Multimodal Large Language Models: A Survey

Awesome_LLM_System-PaperList

Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of papers on accelerating LLMs, currently focusing mainly on inference acceleration, and related works will be gradually added in the future. Welcome contributions!

Awesome-LLM-Long-Context-Modeling

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

llm-inference-benchmark

LLM Inference benchmark

SpeculativeDecodingPapers

📰 Must-read papers and blogs on Speculative Decoding ⚡️

lectures

Material for cuda-mode lectures

LLM101n

LLM101n: Let's build a Storyteller

Awesome-Efficient-LLM

A curated list for Efficient Large Language Models