speculative-decoding

There are 0 repository under speculative-decoding topic.

intel / intel-extension-for-transformers
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
large-language-model chatbot 4-bits llm-inference llm-cpu chatpdf streamingllm intel-optimized-llamacpp speculative-decoding neural-chat habana neural-chat-7b rag retrieval autoround gaudi3
Language:Python 1991
SafeAILab / EAGLE
[ICML'24] EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
large-language-models llm-inference speculative-decoding
Language:Python 532
Infini-AI-Lab / Sequoia
scalable and robust tree-based speculative decoding algorithm
efficiency inference llm speculative-decoding
Language:Python 263
Infini-AI-Lab / TriForce
TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
acceleration llm long-context speculative-decoding llm-inference efficiency inference
Language:Python 124
kssteven418 / BigLittleDecoder
[NeurIPS'23] Speculative Decoding with Big Little Decoder
decoding efficient-inference fast-inference llm speculative-execution speculative-decoding
Language:Python 82
hemingkx / SpecDec
Codes for our paper "Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation" (EMNLP 2023 Findings)
speculative-decoding non-autoregressive
Language:Python 26
mscheong01 / speculative_decoding.c
minimal C implementation of speculative decoding based on llama2.c
artificial-intelligence c llama2 llm speculative-decoding
Language:C 14
AutonomicPerfectionist / PipeInfer
PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation
inference llamacpp llm speculative-decoding
Language:C++ 1
pinqian77 / Dynasurge
Dynasurge: Dynamic Tree Speculation for Prompt-Specific Decoding
large-language-models speculative-decoding
Language:Python 1
romsto / Speculative-Decoding
Implementation of the paper Fast Inference from Transformers via Speculative Decoding, Leviathan et al. 2023.
fast-inference llm llm-inference llm-optimization speculative-decoding
Language:Python 1
u-hyszk / japanese-speculative-decoding
Verification of the effect of speculative decoding in Japanese.
fast-inference japanese nlp speculative-decoding
Language:Python 1

speculative-decoding

intel / intel-extension-for-transformers

SafeAILab / EAGLE

Infini-AI-Lab / Sequoia

Infini-AI-Lab / TriForce

kssteven418 / BigLittleDecoder

hemingkx / SpecDec

mscheong01 / speculative_decoding.c

AutonomicPerfectionist / PipeInfer

pinqian77 / Dynasurge

romsto / Speculative-Decoding

u-hyszk / japanese-speculative-decoding