akkikiki

function_vectors

Function Vectors in Large Language Models (ICLR 2024)

nanotron

Minimalistic large language model 3D-parallelism training

sae

Sparse autoencoders

mergekit

Tools for merging pretrained large language models.

ultravox

SAELens

Training Sparse Autoencoders on Language Models

MLLM-Judge

[ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.

template

This is the repository for the distill web framework

parallelizing_linear_rnns

mamba-triton

recurrentgemma

Open weights language model from Google DeepMind, based on Griffin.

deep_learning_curriculum

Language model alignment-focused deep learning curriculum

Triton-Puzzles

Puzzles for learning Triton

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

Vim

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

mamba.py

A simple and efficient Mamba implementation in pure PyTorch and MLX.

kotomamba

Mamba training library developed by kotoba technologies

mamba

Mamba SSM architecture

alignment-handbook

Robust recipes to align language models with human and AI preferences

gpt_paper_assistant

GPT4 based personalized ArXiv paper assistant bot

trl

Train transformer language models with reinforcement learning.

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

train-with-fsdp

mistral-inference

Official inference library for Mistral models

open_lm

A repository for research on medium sized language models.

ocr-post-correction

CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

shell-ai

LangChain powered shell command generator and runner CLI