janicevidal

llama.cpp

LLM inference in C/C++

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

llama2.c

Inference Llama 2 in one file of pure C

LLMSurvey

The official GitHub page for the survey paper "A Survey of Large Language Models".

TensorRT-LLM

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.

GroundingDINO

Official implementation of the paper "Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection"

tvm_mlir_learn

compiler learning resources collect.

Awesome-LLM-Inference

📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.

py-motmetrics

:bar_chart: Benchmark multiple object trackers (MOT) in Python

LookaheadDecoding

tph-yolov5

InferLLM

a lightweight LLM model inference framework

Conformer

Official code for Conformer: Local Features Coupling Global Representations for Visual Recognition

distrifuser

[CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

TinySAM

Official PyTorch implementation of "TinySAM: Pushing the Envelope for Efficient Segment Anything Model"

HorNet

[NeurIPS 2022] HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions

FaceStudio

Put Your Face Everywhere in Seconds.

SlimSAM

SlimSAM: 0.1% Data Makes Segment Anything Slim

ViT-CoMer

Official implementation of the CVPR 2024 paper ViT-CoMer: Vision Transformer with Convolutional Multi-scale Feature Interaction for Dense Predictions.

CAMixerSR

CAMixerSR: Only Details Need More “Attention” (CVPR 2024)

TensorRT_Inference_Demo

A repo that uses TensorRT to deploy wll-trained models.Support RTDETR,YOLO-NAS,YOLOV5,YOLOV6,YOLOV7,YOLOV8,YOLOX.

yolox-pytorch

a Pytorch easy re-implement of "YOLOX: Exceeding YOLO Series in 2021"

Samba

Effective-Fusion-Factor

Effective Fusion Factor in FPN for Tiny Object Detection(WACV2021)

u-mixformer

OpenMMLab Semantic Segmentation Toolbox and Benchmark.

TrackDiffusion

Official PyTorch implementation of TrackDiffusion (https://arxiv.org/abs/2312.00651)

ITER

PyTorch codes for "Iterative Token Evaluation and Refinement for Real-World Super-Resolution", AAAI 2024

ICELUT

Taming Lookup Tables for Efficient Image Retouching

Hetu-Galvatron

Galvatron is an automatic distributed training system designed for Transformer models, including Large Language Models (LLMs).

DHU-MMCT

Towards Effective Multi-Moving Camera Tracking: A New Dataset and Lightweight Link Model