apoorvagnihotri

Apoorv Agnihotri's starred repositories

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

Language:PythonApache-2.02352600

course

The Hugging Face course on Transformers

Language:MDXApache-2.0207900

accelerate

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

Language:PythonApache-2.0742500

SqueezeNet

SqueezeNet: AlexNet-level accuracy with 50x fewer parameters

BSD-2-Clause216500

litgpt

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

Language:PythonApache-2.0901200

minillm

MiniLLM is a minimal system for running modern LLMs on consumer-grade GPUs

Language:PythonMIT83800

qserve

QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving

Language:PythonApache-2.035300

marlin

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

Language:PythonApache-2.047100

AutoGPTQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Language:PythonMIT415200

tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators

Language:PythonApache-2.01144900

triton

Development repository for the Triton language and compiler

Language:C++MIT1206700

:robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

Language:C++MIT2208400