akkikiki

nanoGPT

The simplest, fastest repository for training/finetuning medium-sized GPTs.

LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

mamba

Mamba SSM architecture

llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

ggml

Tensor library for machine learning

mistral-src

Reference implementation of Mistral AI 7B v0.1 model.

trl

Train transformer language models with reinforcement learning.

DiT

Official PyTorch Implementation of "Scalable Diffusion Models with Transformers"

composer

Supercharge Your Model Training

alignment-handbook

Robust recipes to align language models with human and AI preferences

Vim

Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

mamba-minimal

Simple, minimal implementation of the Mamba SSM in one file of PyTorch.

mPLUG-Owl

mPLUG-Owl & mPLUG-Owl2: Modularized Multimodal Large Language Model

deep_learning_curriculum

Language model alignment-focused deep learning curriculum

shell-ai

LangChain powered shell command generator and runner CLI

summarize-from-feedback

Code for "Learning to summarize from human feedback"

Triton-Puzzles

Puzzles for learning Triton

RRHF

[NIPS2023] RRHF & Wombat

mamba.py

A simple and efficient Mamba implementation in PyTorch and MLX.

gpt_paper_assistant

GPT4 based personalized ArXiv paper assistant bot

open_lm

A repository for research on medium sized language models.

CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

ocr-post-correction

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

train-with-fsdp

mBLIP

deepform

Experimental form data extraction for journalism

kotomamba

Mamba training library developed by kotoba technologies

rl4f

Code for RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs. ACL 2023.

xtreme-up