proger

ml-planner

RLHF-Reward-Modeling

Recipes to train reward model for RLHF.

lightning-thunder

Make PyTorch models up to 40% faster! Thunder is a source to source compiler for PyTorch. It enables using different hardware executors at once; across one or thousands of GPUs.

isat

In situ adaptive tabulation (ISAT) python implementation

locost-summarization

ozIMMU

FP64 equivalent GEMM via Int8 Tensor Cores using the Ozaki scheme

geotorch

Constrained optimization toolkit for PyTorch

FIND

FIND: search For Inductive biases IN Deep seq2seq

neural_testbed

enn

big_vision

Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.

NumericsOfML

Notes for the Numerics of Machine Learning Lecture Course at the University of Tübingen

grok

HVM

A massively parallel, optimal functional runtime in Rust

snac

Multi-Scale Neural Audio Codec (SNAC) compresses audio into discrete codes at a low bitrate

Next-Token-Failures

transformer-debugger

GEAR

GEAR: An Efficient KV Cache Compression Recipefor Near-Lossless Generative Inference of LLM

bayesian_flow_networks

A PyTorch implementation of Bayesian flow networks (Graves et al., 2023).

bayesian_lora

Bayesian Low-Rank Adaptation for Large Language Models

ryg_rans

Simple rANS encoder/decoder (arithmetic coding-ish entropy coder).

flora-opt

This is the official repository for the paper "Flora: Low-Rank Adapters Are Secretly Gradient Compressors" in ICML 2024.

jax-sph

JAX-SPH: A Differentiable Smoothed Particle Hydrodynamics Framework

autocrit

A repository for transformer critique learning and generation

GaLore

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

asdl

ASDL: Automatic Second-order Differentiation Library for PyTorch

fltr

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

mlx-assistant

A simple script to enhance text editing across your Mac, leveraging the power of MLX. Designed for seamless integration, it offers real-time text correction everywhere you type.

HiddenMambaAttn

Official PyTorch Implementation of "The Hidden Attention of Mamba Models"

Score-Entropy-Discrete-Diffusion

[ICML 2024 Oral] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)