janicevidal

llama.cpp

LLM inference in C/C++

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

llama2.c

Inference Llama 2 in one file of pure C

tvm_mlir_learn

compiler learning resources collect.

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

MambaOut

MambaOut: Do We Really Need Mamba for Vision?

ppl.nn

A primitive library for neural network

LookaheadDecoding

tph-yolov5

InferLLM

a lightweight LLM model inference framework

Conformer

Official code for Conformer: Local Features Coupling Global Representations for Visual Recognition

distrifuser

[CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

rtp-llm

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

TinySAM

Official PyTorch implementation of "TinySAM: Pushing the Envelope for Efficient Segment Anything Model"

HorNet

[NeurIPS 2022] HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

SlimSAM

SlimSAM: 0.1% Data Makes Segment Anything Slim

KVQuant

KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization

inferflow

Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).

ViT-CoMer

Official implementation of the CVPR 2024 paper ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions.

CAMixerSR

CAMixerSR: Only Details Need More “Attention” (CVPR 2024)

dash-infer

DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including x86 and ARMv9.

PipeFusion

A Suite of Parallel Approaches for Inference of Diffusion Transformer Models on GPU Clusters

Samba

LW-DETR

This repository is an official implementation of the paper "LW-DETR: A Transformer Replacement to YOLO for Real-Time Detection".

Effective-Fusion-Factor

Effective Fusion Factor in FPN for Tiny Object Detection(WACV2021)

u-mixformer

OpenMMLab Semantic Segmentation Toolbox and Benchmark.

ITER

PyTorch codes for "Iterative Token Evaluation and Refinement for Real-World Super-Resolution", AAAI 2024

ICELUT

Taming Lookup Tables for Efficient Image Retouching

Hetu-Galvatron

Galvatron is an automatic distributed training system designed for Transformer models, including Large Language Models (LLMs).

DHU-MMCT

Towards Effective Multi-Moving Camera Tracking: A New Dataset and Lightweight Link Model