akkikiki

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

mamba

Mamba SSM architecture

mistral-inference

Official inference library for Mistral models

trl

Train transformer language models with reinforcement learning.

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

alignment-handbook

Robust recipes to align language models with human and AI preferences

mergekit

Tools for merging pretrained large language models.

Vim

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

deep_learning_curriculum

Language model alignment-focused deep learning curriculum

nanotron

Minimalistic large language model 3D-parallelism training

Triton-Puzzles

Puzzles for learning Triton

mamba.py

A simple and efficient Mamba implementation in pure PyTorch and MLX.

ultravox

A fast multimodal LLM for real-time voice

template

This is the repository for the distill web framework

recurrentgemma

Open weights language model from Google DeepMind, based on Griffin.

gpt_paper_assistant

GPT4 based personalized ArXiv paper assistant bot

open_lm

A repository for research on medium sized language models.

SAELens

Training Sparse Autoencoders on Language Models

sae

Sparse autoencoders

CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

blogcaster

Python tools for easily translating your blog content to podcasts & YouTube

ocr-post-correction

function_vectors

Function Vectors in Large Language Models (ICLR 2024)

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

train-with-fsdp

kotomamba

Mamba training library developed by kotoba technologies

MLLM-Judge

[ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.

parallelizing_linear_rnns

mamba-triton