ChenMnZ

LLaMA-Factory

A WebUI for Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

llama3

The official Meta Llama 3 GitHub site

Open-Sora-Plan

This project aim to reproduce Sora (Open AI T2V model), we wish the open source community contribute to this project.

FlexGen

Running large language models on a single GPU for throughput-oriented scenarios.

OLMo

Modeling, training, eval, and inference code for OLMo

kimi-free-api

🚀 KIMI AI 长文本大模型逆向API白嫖测试【特长：长文本解读整理】，支持高速流式输出、智能体对话、联网搜索、长文档解读、图像OCR、多轮对话，零配置部署，多路token支持，自动清理会话痕迹。

LLMDataHub

A quick guide (especially) for trending instruction finetuning datasets

executorch

On-device AI across mobile, embedded and edge for PyTorch

BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Efficient-LLMs-Survey

[TMLR 2024] Efficient Large Language Models: A Survey

VILA

VILA - a multi-image visual language model with training, inference and evaluation recipe, deployable from cloud to edge (Jetson Orin and laptops)

optimum-quanto

A pytorch quantization backend for optimum

sparsegpt

Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

MobiLlama

MobiLlama : Small Language Model tailored for edge devices

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

BitBLAS

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

Atom

[MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving

QuaRot

Code for QuaRot, an end-to-end 4-bit inference of large language models.

Incremental-Network-Quantization

Caffe Implementation for Incremental network quantization

KIVI

KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cache

BitMat

An efficent implementation of the method proposed in "The Era of 1-bit LLMs"

QUICK

QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inference

lmquant

MMT-Bench

ICML'2024 | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

fast-hadamard-transform

Fast Hadamard transform in CUDA, with a PyTorch interface

bllama

1.58-bit LLaMa model

LLaVA-PruMerge

LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models

svit

Official implementation of "SViT: Revisiting Token Pruning for Object Detection and Instance Segmentation"

gptvq

qat-pretrain