nawnoes

ReST-MCTS

ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search

RLHF-Reward-Modeling

Recipes to train reward model for RLHF.

prepacking

The source code of our work "Prepacking: A Simple Method for Fast Prefilling and Increased Throughput in Large Language Models"

penzai

A JAX research toolkit for building, editing, and visualizing neural networks.

simple-evals

schedule_free

Schedule-Free Optimization in PyTorch

LogicKor

한국어 언어모델 다분야 사고력 벤치마크

terashuf

terashuf shuffles multi-terabyte text files using limited memory

RingAttention

Transformers with Arbitrarily Large Context

ring-flash-attention

Ring attention implementation with flash attention

Inflection-Benchmarks

Public Inflection Benchmarks

transformer-debugger

ko-rm-judge

Reward Model을 이용하여 언어모델의 답변을 평가하기

DataDreamer

DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models.   🤖💤

gemma_pytorch

The official PyTorch implementation of Google's Gemma models

large_language_model_training_playbook

An open collection of implementation tips, tricks and resources for training large language models

langroid

Harness LLMs with Multi-Agent Programming

awesome-llm-powered-agent

Awesome things about LLM-powered agents. Papers / Repos / Blogs / ...

textbook_quality

Generate textbook-quality synthetic LLM pretraining data

InternEvo

RethinkTinyLM

The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”

SPIN

The official implementation of Self-Play Fine-Tuning (SPIN)

LongBench

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding

LongAlign

LongAlign: A Recipe for Long Context Alignment Encompassing Data, Training, and Evaluation

BlackMamba

Code repository for Black Mamba

sparse_feedback

miracl

A large-scale multilingual dataset for Information Retrieval. Thorough human-annotations across 18 diverse languages.

nanotron

Minimalistic large language model 3D-parallelism training

ML-Papers-of-the-Week

🔥Highlighting the top ML papers every week.

scalax

A simple library for scaling up JAX programs