efficient-inference

There are 2 repositories under efficient-inference topic.

huawei-noah / Efficient-AI-Backbones
Efficient AI Backbones including GhostNet, TNT and MLP, developed by Huawei Noah's Ark Lab.
convolutional-neural-networks efficient-inference ghostnet imagenet model-compression pretrained-models pytorch tensorflow transformer vision-transformer
Language:Python 4328
SqueezeAILab / LLMCompiler
[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
function-calling llm llm-agent llm-agents llms parallel-function-call efficient-inference large-language-models llama llama2 llm-framework natural-language-processing nlp transformer
Language:Python 1778
snap-research / EfficientFormer
EfficientFormerV2 [ICCV 2023] & EfficientFormer [NeurIPs 2022]
deep-learning detection efficient-inference efficient-neural-networks pytorch semantic-segmentation transformer imagenet transformers mobile-devices
Language:Python 1085
huawei-noah / AdderNet
Code for paper " AdderNet: Do We Really Need Multiplications in Deep Learning?"
pytorch imagenet convolutional-neural-networks cvpr2020 efficient-inference
Language:Python 966
horseee / DeepCache
[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free
diffusion-models efficient-inference model-compression stable-diffusion training-free
Language:Python 939
Zhen-Dong / Awesome-Quantization-Papers
List of papers related to neural network quantization in recent AI conferences and journals.
quantization awesome-list papers neural-networks model-compression edge-computing efficient-inference diffusion-models large-language-models
752
VITA-Group / LightGaussian
[NeurIPS 2024 Spotlight]"LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS", Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, Zhangyang Wang
3d-reconstruction efficient-inference gaussian-splatting neurips-2024 nurips
Language:Python 742
SqueezeAILab / SqueezeLLM
[ICML 2024] SqueezeLLM: Dense-and-Sparse Quantization
efficient-inference large-language-models llm model-compression natural-language-processing post-training-quantization quantization text-generation transformer llama localllm small-models
Language:Python 705
liuzhuang13 / slimming
Learning Efficient Convolutional Networks through Network Slimming, In ICCV 2017.
convolutional-neural-networks deep-learning efficient-inference
Language:Lua 572
SqueezeAILab / KVQuant
[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
compression efficient-inference efficient-model large-language-models llama llm localllama localllm mistral model-compression natural-language-processing quantization small-models text-generation transformer
Language:Python 390
xuyang-liu16 / Awesome-Generation-Acceleration
📚 Collection of awesome generation acceleration resources.
diffusion-models efficient-deep-learning efficient-inference model-acceleration text-to-image text-to-video image-generation video-generation
361
lucidrains / speculative-decoding
Explorations into some recent techniques surrounding speculative decoding
artificial-intelligence deep-learning efficient-inference transformers
Language:Python 290
Picovoice / picollm
On-device LLM Inference Powered by X-Bit Quantization
compression efficient-inference gemma generative-ai language-model language-models large-language-model llama llama2 llama3 llm llm-inference llms mistral mixtral model-compression natural-language-processing quantization self-hosted
Language:Python 272
SYSU-SAIL / SMSR
[CVPR 2021] Exploring Sparsity in Image Super-Resolution for Efficient Inference
super-resolution sparsity efficient-inference
Language:Python 239
xindongzhang / ELAN
[ECCV2022] Efficient Long-Range Attention Network for Image Super-resolution
efficient-inference super-resolution transformer
Language:Python 230
changlin31 / DS-Net
(CVPR 2021, Oral) Dynamic Slimmable Network
dynamic-networks pruning network-pruning dynamic-pruning model-compression efficient-inference
Language:Python 229
czg1225 / AsyncDiff
[NeurIPS 2024] AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising
diffusion-models distributed-computing inference-acceleration training-free efficient-inference stable-diffusion text-to-image text-to-video
Language:Python 206
liuziwei7 / mobile-id
Deep Face Model Compression
computer-vision deep-learning efficient-inference face-recognition model-compression
Language:MATLAB 195
cure-lab / DeciWatch
[ECCV 2022] Official implementation of the paper "DeciWatch: A Simple Baseline for 10x Efficient 2D and 3D Pose Estimation"
2d-human-pose 3d-pose-estimation body-reconstruction efficient-inference human-pose-estimation 3d-body-recovery deep-learning efficiency efficient-neural-networks pose-estimation pytorch eccv eccv2022
Language:Python 185
SimonAytes / SoT
Official code repository for Sketch-of-Thought (SoT)
ai efficient-inference llm llm-inference prompting
Language:Python 129
horseee / learning-to-cache
[NeurIPS 2024] Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching
diffusion-models efficient-inference
Language:Python 116
kssteven418 / BigLittleDecoder
[NeurIPS'23] Speculative Decoding with Big Little Decoder
decoding efficient-inference fast-inference llm speculative-execution speculative-decoding
Language:Python 95
snap-research / graphless-neural-networks
[ICLR 2022] Code for Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation (GLNN)
deep-learning distillation efficient-inference graph-algorithm graph-neural-networks knowledge-distillation pytorch gnn scalability
Language:Python 95
RAIVNLab / STR
Soft Threshold Weight Reparameterization for Learnable Sparsity
sparsity learnable-sparsity sparsity-optimization cnn efficient-inference edge-machine-learning soft-thresholding str imagenet resource-efficient icml-2020 icml icml2020 soft-threshold-reparameterization
Language:Python 90
Alpha-Innovator / AdaptiveDiffusion
[NeurIPS'24] Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy
adaptive-inference diffusion-models efficient-inference model-acceleration stable-diffusion training-free
Language:Python 72
qiuk2 / AAR
[Official Implementation] Acoustic Autoregressive Modeling 🔥
efficient-inference audio-tokenizer autoregressive-generation next-scale-prediction
Language:Python 72
raymin0223 / fast_robust_early_exit
Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decoding (EMNLP 2023 Long)
autoregressive-models early-exiting efficient-inference nlp llms
Language:Python 64
FranxYao / Partially-Observed-TreeCRFs
Implementation of AAAI 21 paper: Nested Named Entity Recognition with Partially Observed TreeCRFs
crf named-entity-recognition efficient-inference nested-named-entity-recognition tree-crf tree-structure sum-product-algorithm sum-product
Language:Python 51
IBM / AdaMML
Official implementation of AdaMML. https://arxiv.org/abs/2105.05165.
computer-vision multimodal-learning deep-learning efficient-inference
Language:Python 51
snu-mllab / GuidedQuant
Official PyTorch implementation of "GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance" (ICML 2025)
efficient-inference large-language-models llm-inference llm-quantization quantization
Language:Python 47
tchittesh / lzu
Code for Learning to Zoom and Unzoom (CVPR 2023)
3d-detection autonomous-driving efficient-inference spatial-attention
Language:Python 46
LiuHengyu321 / FlexGS
[CVPR2025] Code Release for "FlexGS: Train Once, Deploy Everywhere with Many-in-One Flexible 3D Gaussian Splatting"
gaussian-splatting novel-view-synthesis 3d-reconstruction efficient-inference
Language:Python 43
ivclab / agegenderLMTCNN
Jia-Hong Lee, Yi-Ming Chan, Ting-Yen Chen, and Chu-Song Chen, "Joint Estimation of Age and Gender from Unconstrained Face Images using Lightweight Multi-task CNN for Mobile Applications," IEEE International Conference on Multimedia Information Processing and Retrieval, MIPR 2018
age-gender-cnn mobile-application multi-task-learning tensorflow android-application efficient-inference deep-neural-networks
Language:Python 40
yikaiw / RS-Nets
[ECCV 2020] Code release for "Resolution Switchable Networks for Runtime Efficient Image Recognition"
distillation eccv2020 efficient-inference ensemble multi-resolution quantization rsnet switchable
Language:Python 40
bharathsudharsan / TinyML-Benchmark-NNs-on-MCUs
Code for WF-IoT paper 'TinyML Benchmark: Executing Fully Connected Neural Networks on Commodity Microcontrollers'
tinyml-benchmark raspberry-pi-pico mcu-boards arduinio armcortexm0 armcortexm4 armcortexm7 machine-learning tinyml efficient-inference tflite tfmicro c-code-generator cmsis-nn
Language:Python 37
xuyang-liu16 / VidCom2
🚀 Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Models
efficient-inference large-language-model large-vision-language-models video-large-language-models
Language:Python 37