akkikiki

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

mamba

Mamba SSM architecture

mistral-inference

Official inference library for Mistral models

trl

Train transformer language models with reinforcement learning.

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

alignment-handbook

Robust recipes to align language models with human and AI preferences

mergekit

Tools for merging pretrained large language models.

Vim

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

deep_learning_curriculum

Language model alignment-focused deep learning curriculum

nanotron

Minimalistic large language model 3D-parallelism training

shell-ai

LangChain powered shell command generator and runner CLI

Triton-Puzzles

Puzzles for learning Triton

mamba.py

A simple and efficient Mamba implementation in pure PyTorch and MLX.

template

This is the repository for the distill web framework

ultravox

recurrentgemma

Open weights language model from Google DeepMind, based on Griffin.

gpt_paper_assistant

GPT4 based personalized ArXiv paper assistant bot

open_lm

A repository for research on medium sized language models.

SAELens

Training Sparse Autoencoders on Language Models

CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

sae

Sparse autoencoders

ocr-post-correction

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

train-with-fsdp

function_vectors

Function Vectors in Large Language Models (ICLR 2024)

kotomamba

Mamba training library developed by kotoba technologies

parallelizing_linear_rnns

mamba-triton

MLLM-Judge

[ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.