Kazuki Fujii's repositories
moe-recipes
Ongoing research training Mixture of Expert models.
llm-recipes
Ongoing Research Project for continaual pre-training LLM(dense mode)
wandb_watcher
ABCI 大規模言語モデル構築支援にてwandbのジョブを監視するためのツール
Megatron-LM
Ongoing research training transformer models at scale
deploymentmanager-samples
Deployment Manager samples and templates.
grouped_gemm
PyTorch bindings for CUTLASS grouped GEMM.
levanter
Legible, Scalable, Reproducible Foundation Models with Named Tensors and Jax
llama-recipes
Examples and recipes for Llama 2 model
llama3v
A SOTA vision model built on top of llama3 8B.
Megatron-LM-ABCI
NVIDIA Megatron-LM fork
ml-engineering
Machine Learning Engineering Open Book
nanotron
Minimalistic large language model 3D-parallelism training
NeMo
NeMo: a toolkit for conversational AI
NeMo-Aligner
Scalable toolkit for efficient model alignment
NeMo-Megatron-Launcher
NeMo Megatron launcher and tools
torchtitan
A native PyTorch Library for large model training
TransformerEngine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.