aigaolc

LLM101n

LLM101n: Let's build a Storyteller

Qwen2

Qwen2 is the large language model series developed by Qwen team, Alibaba Cloud.

DiffSynth-Studio

Enjoy the magic of Diffusion models!

gemma.cpp

lightweight, standalone C++ inference engine for Google's Gemma models.

gemma_pytorch

The official PyTorch implementation of Google's Gemma models

CosyVoice

Multi-lingual large voice generation model, providing inference, training and deployment full-stack ability.

anthropic-cookbook

A collection of notebooks/recipes showcasing some fun and effective ways of using Claude.

swift

ms-swift: Use PEFT or Full-parameter to finetune 300+ LLMs or 50+ MLLMs. (Qwen2, GLM4v, Internlm2.5, Yi, Llama3.1, Llava-Video, Internvl2, MiniCPM-V-2.6, Deepseek, Baichuan2, Gemma2, Phi3-Vision, ...)

SenseVoice

Multilingual Voice Understanding Model

data-juicer

A one-stop data processing system to make data higher-quality, juicier, and more digestible for (multimodal) LLMs! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷为大模型提供更高质量、更丰富、更易”消化“的数据！

optax

Optax is a gradient processing and optimization library for JAX.

maxtext

A simple, performant and scalable Jax LLM!

lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval

dclm

DataComp for Language Models

deduplicate-text-datasets

Mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

YouTokenToMe

Unsupervised text tokenizer focused on computational efficiency

MINT-1T

MINT-1T: A one trillion token multimodal interleaved dataset.

lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

tensorrtllm_backend

The Triton TensorRT-LLM Backend

self-refine

LLMs can generate feedback on their work, use it to improve the output, and repeat this process iteratively.

H3

Language Modeling with the H3 State Space Model

gemma-cookbook

A collection of guides and examples for the Gemma open models from Google.

open_lm

A repository for research on medium sized language models.

FuseAI

FuseAI Project

torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

bigcodebench

BigCodeBench: Benchmarking Code Generation Towards AGI

eval-scope

A streamlined and customizable framework for efficient large model evaluation and performance benchmarking

DRTK

Differentiable Rendering Toolkit

Zyda_processing