wpybtw

llm.c

LLM training in simple, raw C/CUDA

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

cosmopolitan

build-once run-anywhere c library

llamafile

Distribute and run LLMs with a single file.

ragflow

RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

gperftools

Main gperftools repository

lm-evaluation-harness

A framework for few-shot evaluation of language models.

transformers_tasks

⭐️ NLP Algorithms with transformers lib. Supporting Text-Classification, Text-Generation, Information-Extraction, Text-Matching, RLHF, SFT etc.

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

TransformerEngine

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.

yarn

YaRN: Efficient Context Window Extension of Large Language Models

Awesome-Efficient-LLM

A curated list for Efficient Large Language Models

tutel

Tutel MoE: An Optimized Mixture-of-Experts Implementation

cpufp

A CPU tool for benchmarking the peak of floating points

nvbench

CUDA Kernel Benchmarking Library

ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

H2O

[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.

NBCE

Naive Bayes-based Context Extension

DejaVu

infini-transformer

PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)

KsanaLLM

turingas

Assembler for NVIDIA Volta and Turing GPUs

LM-Infinite

Implementation of paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"

Parallel-Context-Windows

stk

pyaskit

AskIt: Unified programming interface for programming with LLMs (GPT-3.5, GPT-4, Gemini, Claude, Cohere, Llama 2)

gpu-arch-microbenchmark

Dissecting NVIDIA GPU Architecture

llmperf

LLMCompass_ISCA_AE