zw123han

picoGPT

An unnecessarily tiny implementation of GPT-2 in NumPy.

TTS

🐸💬 - a deep learning toolkit for Text-to-Speech, battle-tested in research and production

ml-engineering

Machine Learning Engineering Open Book

streaming-llm

[ICLR 2024] Efficient Streaming Language Models with Attention Sinks

headshots-starter

LLaMA-Factory

Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

text-generation-inference

Large Language Model Text Generation Inference

audiocraft

Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

llm-attacks

Universal and Transferable Attacks on Aligned Language Models

so-vits-svc-fork

so-vits-svc fork with realtime support, improved interface and more features.

so-vits-svc

SoftVC VITS Singing Voice Conversion

tinygrad

You like pytorch? You like micrograd? You love tinygrad! ❤️

deep-language-networks

We view Large Language Models as stochastic language layers in a network, where the learnable parameters are the natural language prompts at each layer. We stack two such layers, feeding the output of one layer to the next. We call the stacked architecture a Deep Language Network - DLN

whisper

Robust Speech Recognition via Large-Scale Weak Supervision

CodeGeeX

CodeGeeX: An Open Multilingual Code Generation Model (KDD 2023)

evals

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

alpaca-lora

Instruct-tune LLaMA on consumer hardware

alpaca.cpp

Locally run an Instruction-Tuned Chat-Style LLM

llama.cpp

LLM inference in C/C++

JAX_llama

Inference code for LLaMA models in JAX

stanford_alpaca

Code and documentation to train Stanford's Alpaca models, and generate the data.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

unnatural-instructions

paxml

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelization, and has demonstrated industry leading model flop utilization rates.

levanter

Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax

flash-attention-jax

Implementation of Flash Attention in Jax

workshops

This is a repository for all workshop related materials.

PiPPy

Pipeline Parallelism for PyTorch

tuning_playbook

A playbook for systematically maximizing the performance of deep learning models.

jax-llm-benchmarking

Scripts for benchmarking LLM fine-tuning throughput.