flyinglandlord

awesome-model-quantization

A list of papers, docs, codes about model quantization. This repo is aimed to provide the info for model quantization research, we are continuously improving the project. Welcome to PR the works (papers, repositories) that are missed by the repo.

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

latent-diffusion

High-Resolution Image Synthesis with Latent Diffusion Models

DenoisingDiffusionProbabilityModel-ddpm-

This may be the simplest implement of DDPM. You can directly run Main.py to train the UNet on CIFAR-10 dataset and see the amazing process of denoising.

LLM-Viewer

Analyze the inference of Large Language Models (LLMs). Analyze aspects like computation, storage, transmission, and hardware roofline model in a user-friendly interface.

llmc

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

KVQuant

[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

ComputeLibrary

The Compute Library is a set of computer vision and machine learning functions optimised for both Arm CPUs and GPUs using SIMD technologies.

EasyLLM

Built upon Megatron-Deepspeed and HuggingFace Trainer, EasyLLM has reorganized the code logic with a focus on usability. While enhancing usability, it also ensures training efficiency.

S-LoRA

S-LoRA: Serving Thousands of Concurrent LoRA Adapters

lightllm

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

awesome-lm-system

Summary of system papers/frameworks/codes/tools on training or serving large model

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

text-generation-inference

Large Language Model Text Generation Inference

mlc-llm

Universal LLM Deployment Engine with ML Compilation

FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

BUAA-CT-2022

北航编译技术2022

CSYuTuiMian2023

LogicStack-LeetCode

公众号「宫水三叶的刷题日记」刷穿 LeetCode 系列文章源码

Universe

哔哩哔哩流媒体平台功能增强系列模块

CSSummerCamp2023

boat4study-release-page

学舟平台发布页

iRingo

解锁完整的 Apple功能和集成服务

boat4study_frontend

学舟平台前端——自适应Web-App和微信小程序

barracuda-frontend

Database Group Homework

playerdemo

一个视频播放器，开源版 potplayer ，用于总结播放器开发技术。