ShuoTang123

MATRIX

Implementation of the MATRIX framework (ICML 2024)

DeLAMA

O-LoRA

ContrastiveDecoding

contrastive decoding

FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

OpenFedLLM

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

llm-continual-learning-survey

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

self-speculative-decoding

Code associated with the paper **Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**

social-media-profile-scrapers

Fetch user's data across social media