MitchellX

glake

GLake: optimizing GPU memory management and IO transmission.

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

PowerInfer

High-speed Large Language Model Serving on PCs with Consumer-grade GPUs

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

bilivideos

flash-llm

Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity

flash-attention

Fast and memory-efficient exact attention

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

autoLiterature

autoLiterature是一个基于Python的自动文献管理命令行工具

Awesome-Multimodal-Large-Language-Models

:sparkles::sparkles:Latest Papers and Datasets on Multimodal Large Language Models, and Their Evaluation.

mPLUG-DocOwl

mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

MobiVQA

Transformers-Tutorials

This repository contains demos I made with the Transformers library by HuggingFace.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

donut

Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022

text-to-text-transfer-transformer

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

t5x

pix2struct

awesome-mixture-of-experts

A collection of AWESOME things about mixture-of-experts

Awesome-Mixture-of-Experts-Papers

A curated reading list of research in Mixture-of-Experts(MoE).

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

flex-dm

Towards Flexible Multi-modal Document Models [Inoue+, CVPR2023]

flops-counter.pytorch

Flops counter for convolutional networks in pytorch framework

M-FAC

Efficient reference implementations of the static & dynamic M-FAC algorithms (for pruning and optimization)

rigl

End-to-end training of sparse deep neural networks with little-to-no performance loss.

snip

Pytorch implementation of the paper "SNIP: Single-shot Network Pruning based on Connection Sensitivity" by Lee et al.

ColossalAI

Making large AI models cheaper, faster and more accessible

STR

Soft Threshold Weight Reparameterization for Learnable Sparsity

Efficient-Deep-Learning

Collection of recent methods on (deep) neural network compression and acceleration.

hydra

Code and checkpoints of compressed networks for the paper titled "HYDRA: Pruning Adversarially Robust Neural Networks" (NeurIPS 2020) (https://arxiv.org/abs/2002.10509).