Kazuki Fujii's repositories
wandb_watcher
ABCI 大規模言語モデル構築支援にてwandbのジョブを監視するためのツール
accelerate
🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision
OLMo
Modeling, training, eval, and inference code for OLMo
mistral-src
Reference implementation of Mistral AI 7B v0.1 model.
direct-preference-optimization
Reference implementation for DPO (Direct Preference Optimization)
FastChat
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Qwen
The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
nvtop
GPUs process monitoring for AMD, Intel and NVIDIA
Awesome-LLM
Awesome-LLM: a curated list of Large Language Model
epochraft-hf-fsdp
Example of using Epochraft to train HuggingFace transformers models with PyTorch FSDP
gpu-burn
Multi-GPU CUDA stress test
relora
Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
yarn
YaRN: Efficient Context Window Extension of Large Language Models
Yi
A series of large language models trained from scratch by developers @01-ai
open-llms
📋 A list of open LLMs available for commercial use.
m2
Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"
Megatron-LLM
distributed trainer for LLMs
DeepSpeedExamples
Example models using DeepSpeed
open_lm
A repository for research on medium sized language models.
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
attention
several types of attention modules written in PyTorch
python-fire
Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.
simple-simcse-ja
Japanese Simple-SimCSE
fmengine
Utilities for Training Very Large Models
apex
A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
GenerativeImage2Text
GIT: A Generative Image-to-text Transformer for Vision and Language