Yoshinari Fujinuma's starred repositories
llama-recipes
Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.
mistral-src
Reference implementation of Mistral AI 7B v0.1 model.
alignment-handbook
Robust recipes to align language models with human and AI preferences
mamba-minimal
Simple, minimal implementation of the Mamba SSM in one file of PyTorch.
deep_learning_curriculum
Language model alignment-focused deep learning curriculum
summarize-from-feedback
Code for "Learning to summarize from human feedback"
Triton-Puzzles
Puzzles for learning Triton
gpt_paper_assistant
GPT4 based personalized ArXiv paper assistant bot
CoLT5-attention
Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch
mixture-of-attention
Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts