streamingllm

There are 0 repository under streamingllm topic.

intel / intel-extension-for-transformers
⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡
large-language-model chatbot 4-bits llm-inference llm-cpu chatpdf streamingllm intel-optimized-llamacpp speculative-decoding neural-chat habana neural-chat-7b rag retrieval autoround gaudi3
Language:Python 1996
DefTruth / Awesome-LLM-Inference
📖A curated list of Awesome LLM Inference Paper with codes, TensorRT-LLM, vLLM, streaming-llm, AWQ, SmoothQuant, WINT8/4, Continuous Batching, FlashAttention, PagedAttention etc.
flash-attention flash-attention-2 paged-attention streaming-llm tensorrt-llm vllm awesome-llm sora llm llm-inference llms deepseek open-sora
1607

intel / intel-extension-for-transformers