kaix90

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

tinygrad

You like pytorch? You like micrograd? You love tinygrad! ❤️

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Learn-Vim

Learning Vim and Vimscript doesn't have to be hard. This is the guide that you're looking for 📖

AISystem

AISystem 主要是指AI系统，包括AI芯片、AI编译器、AI推理和训练框架等AI全栈底层技术

gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/INT4/FP4/NF4) & sparsity; leading model compression techniques on TensorFlow, PyTorch, and ONNX Runtime

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

Awesome-Efficient-LLM

A curated list for Efficient Large Language Models

Awesome-LLM-Compression

Awesome LLM compression research papers and tools.

Awesome-LLM-Long-Context-Modeling

📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥

yolov5-5.x-annotations

一个基于yolov5-5.0的中文注释版本！

Vehicle-Detection-and-Tracking

Computer vision based vehicle detection and tracking using Tensorflow Object Detection API and Kalman-filtering

SpQR

motpy

Library for tracking-by-detection multi object tracking implemented in python

ao

Custom data types and layouts for training and inference

optimum-intel

🤗 Optimum Intel: Accelerate inference with Intel optimization tools

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

TensorRT-Model-Optimizer

TensorRT Model Optimizer is a unified library of state-of-the-art model optimization techniques such as quantization and sparsity. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM or TensorRT to optimize inference speed on NVIDIA GPUs.

llm-analysis

Latency and Memory Analysis of Transformer Models for Training and Inference

aisys-building-blocks

Building blocks for foundation models.

algorithm-study

Algorithm Notes and Templates (written in python,golang and typescript)

KIVI

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

applied-ai

Applied AI experiments and examples for PyTorch

Machine-Learning-Explained

Learn the theory, math and code behind different machine learning algorithms and techniques.

EasyKV

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

venom

A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores

Sparse-IFT

Official repository of Sparse ISO-FLOP Transformations for Maximizing Training Efficiency

Sparse-GPT-Finetuning

Code for my ICLR 2024 TinyPapers paper "Prune and Tune: Improving Efficient Pruning Techniques for Massive Language Models"