fast-inference

There are 0 repository under fast-inference topic.

foolwood / pytorch-slimming
Learning Efficient Convolutional Networks through Network Slimming, In ICCV 2017.
weight-pruning pytorch l1-regularization fast-inference deep-learning
Language:Python 563
aredden / flux-fp8-api
Flux diffusion model implementation using quantized fp8 matmul & remaining layers use faster half precision accumulate, which is ~2x faster on consumer devices.
diffusion fast-inference flux fp8 pytorch quantization
Language:Python 86
kssteven418 / BigLittleDecoder
[NeurIPS'23] Speculative Decoding with Big Little Decoder
decoding efficient-inference fast-inference llm speculative-execution speculative-decoding
Language:Python 81
dvlab-research / Q-LLM
This is the official repo of "QuickLLaMA: Query-aware Inference Acceleration for Large Language Models"
fast-inference inference-acceleration kv-cache-compression large-language-models long-context
Language:Python 36
romsto / Speculative-Decoding
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
fast-inference llm llm-inference llm-optimization speculative-decoding
Language:Python 12
lim142857 / Sparsifiner
Demo code for CVPR2023 paper "Sparsifiner: Learning Sparse Instance-Dependent Attention for Efficient Vision Transformers"
efficient-transformers efficient-vision-transformers fast-inference sparse-attention attention-mechanism low-rank sparse-neural-networks vision-transformer
Language:Python 11
szemenyeim / RoboDNN
Fast Forward-Only Deep Neural Network Library for the Nao Robots
neural-network nao-robots pruning deep-neural-networks deep-learning library fast-inference robocup
Language:C++ 4
u-hyszk / japanese-speculative-decoding
Verification of the effect of speculative decoding in Japanese.
fast-inference japanese nlp speculative-decoding
Language:Python 2
PopoDev / BiLD
Reproducibility Project for [NeurIPS'23] Speculative Decoding with Big Little Decoder
fast-inference llm speculative-decoding reproducibility
Language:Python 1

fast-inference

foolwood / pytorch-slimming

aredden / flux-fp8-api

kssteven418 / BigLittleDecoder

dvlab-research / Q-LLM

romsto / Speculative-Decoding

lim142857 / Sparsifiner

szemenyeim / RoboDNN

u-hyszk / japanese-speculative-decoding

PopoDev / BiLD