kv-cache
There are 0 repository under kv-cache topic.
- Language:Go 3328
FMInference / H2O
[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
Language:Python 277- Language:Python 62
DRSY / EasyKV
Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)
Language:Python 47hkproj / pytorch-llama-notes
Notes about LLaMA 2 model
Language:Python 28aju22 / LLaMA2
This repository contains an implementation of the LLaMA 2 (Large Language Model Meta AI) model, a Generative Pretrained Transformer (GPT) variant. The implementation focuses on the model architecture and the inference process. The code is restructured and heavily commented to facilitate easy understanding of the key parts of the architecture.
Language:Python 24mehdihosseinimoghadam / AVA-Mistral-7B
Fine-Tuned Mistral 7B Persian Large Language Model LLM / Persian Mistral 7B
Language:Jupyter Notebook 3phkhanhtrinh23 / milliGPT
This a minimal implementation of a GPT model but it has some advanced features such as temperature/ top-k/ top-p sampling, and KV Cache.
Language:Python 1
