pianogGG

rwkv.cpp

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

ppl.nn.llm

gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

mlc-llm

Universal LLM Deployment Engine with ML Compilation

Qwen

The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

Qwen-VL

The official repo of Qwen-VL (通义千问-VL) chat & pretrained large vision language model proposed by Alibaba Cloud.

llama.cpp

LLM inference in C/C++

stable-diffusion-webui

Stable Diffusion web UI

triton

Development repository for the Triton language and compiler

xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.

x-transformers

A simple but complete full-attention transformer with a set of promising experimental features from various papers

lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Video-LLaMA

[EMNLP 2023 Demo] Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

tensorrtllm_backend

The Triton TensorRT-LLM Backend

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

CUDALibrarySamples

CUDA Library Samples

GPTQ-for-LLaMa

4 bits quantization of LLaMA using GPTQ

gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

Train_Transformers_with_INT4

examples

A set of examples around MegEngine

llm-awq

[MLSys 2024 Best Paper Award] AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

ChatGLM-Efficient-Tuning

Fine-tuning ChatGLM-6B with PEFT | 基于 PEFT 的高效 ChatGLM 微调

Baichuan2

A series of large language models developed by Baichuan Intelligent Technology

LangChain-Chinese-Getting-Started-Guide

LangChain 的中文入门教程

bitsandbytes

Accessible large language models via k-bit quantization for PyTorch.

ppl.llm.kernel.cuda

ppl.nn

A primitive library for neural network