There are 0 repository under fast-inference topic.
Learning Efficient Convolutional Networks through Network Slimming, In ICCV 2017.
Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.
[NeurIPS'23] Speculative Decoding with Big Little Decoder
This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
Fast Forward-Only Deep Neural Network Library for the Nao Robots
Verification of the effect of speculative decoding in Japanese.