zfan20

llama.cpp

LLM inference in C/C++

LLM101n

LLM101n: Let's build a Storyteller

llm.c

LLM training in simple, raw C/CUDA

unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators

Perplexica

Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI

ggml

Tensor library for machine learning

tiny-gpu

A minimal GPU design in Verilog to learn how GPUs work from the ground up

ToolBench

[ICLR'24 spotlight] An open platform for training, serving, and evaluating large language model for tool learning.

matmulfreellm

Implementation for MatMul-free LM.

LLM-Agent-Survey

reflexion

[NeurIPS 2023] Reflexion: Language Agents with Verbal Reinforcement Learning

c-style

My favorite C programming practices.

ThunderKittens

Tile primitives for speedy kernels

GaLore

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

distributed-llama

Tensor parallelism is all you need. Run LLMs on an AI cluster at home using any device. Distribute the workload, divide RAM usage, and increase inference speed.

granite-code-models

Granite Code Models: A Family of Open Foundation Models for Code Intelligence

ToolLearningPapers

PiPPy

Pipeline Parallelism for PyTorch

unet.cu

UNet diffusion model in pure CUDA

RepoAgent

An LLM-powered repository agent designed to assist developers and teams in generating documentation and understanding repositories quickly.

Atom

[MLSys'24] Atom: Low-bit Quantization for Efficient and Accurate LLM Serving

awesome-emulators-simulators

A curated list of software emulators and simulators of PCs, home computers, mainframes, consoles, robots and much more...

CEPE

[ACL 2024] Long-Context Language Modeling with Parallel Encodings

Awesome-Mainframes

Awesome list of mainframe related resources & projects

bark.cpp

Port of Suno AI's Bark in C/C++ for fast inference

PLTranslationEmpirical

Artifact repository for the paper "Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code", In Proceedings of The 46th IEEE/ACM International Conference on Software Engineering (ICSE 2024), Lisbon, Portugal, April 2024

llama_duo

asynchronous/distributed speculative evaluation for llama3

Retroformer

proxy_based_uncertainty