janicevidal

alpha-free-matting

onnxsim_large_model

simplify >2GB large onnx model

OutfitAnyone

Outfit Anyone: Ultra-high quality virtual try-on for Any Clothing and Any Person

SimD

LeYOLO

LW-DETR

This repository is an official implementation of the paper "LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection".

dash-infer

DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including x86 and ARMv9.

rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

xDiT

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) on multi-GPU Clusters

KVQuant

KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

MambaOut

MambaOut: Do We Really Need Mamba for Vision?

inferflow

Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

ppl.nn

A primitive library for neural network

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

LookaheadDecoding

[ICML 2024] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding

InferLLM

a lightweight LLM model inference framework

Hetu-Galvatron

Galvatron is an automatic distributed training system designed for Transformer models, including Large Language Models (LLMs).

Samba

ICELUT

[ECCV 2024] Taming Lookup Tables for Efficient Image Retouching

tvm_mlir_learn

compiler learning resources collect.

ViT-CoMer

Official implementation of the CVPR 2024 paper ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions.

CAMixerSR

CAMixerSR: Only Details Need More “Attention” (CVPR 2024)

distrifuser

[CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

llama.cpp

LLM inference in C/C++

Effective-Fusion-Factor

Effective Fusion Factor in FPN for Tiny Object Detection(WACV2021)

HorNet

[NeurIPS 2022] HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

Conformer

Official code for Conformer: Local Features Coupling Global Representations for Visual Recognition

tph-yolov5

llama2.c

Inference Llama 2 in one file of pure C

TinySAM

Official PyTorch implementation of "TinySAM: Pushing the Envelope for Efficient Segment Anything Model"