YudiZh

LLM101n

LLM101n: Let's build a Storyteller

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

llama-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

ml-engineering

Machine Learning Engineering Open Book

Yi

A series of large language models trained from scratch by developers @01-ai

OpenAgents

[COLM 2024] OpenAgents: An Open Platform for Language Agents in the Wild

weak-to-strong

Medusa

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

MotionGPT

[NeurIPS 2023] MotionGPT: Human Motion as a Foreign Language, a unified motion-language generation model using LLMs

MirrorSite

镜像网站合集

MAP-NEO

the-art-of-debugging

The Art of Debugging

EAGLE

Official Implementation of EAGLE-1 and EAGLE-2

PiPPy

Pipeline Parallelism for PyTorch

fairseq2

FAIR Sequence Modeling Toolkit 2

DoLa

Official implementation for the paper "DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models"

ALMA

State-of-the-art LLM-based translation models.

ChunkLlama

[ICML'24] Data and code for our paper "Training-Free Long-Context Scaling of Large Language Models"

Transformer-M

[ICLR 2023] One Transformer Can Understand Both 2D & 3D Molecular Data (official implementation)

Chinese-Tiny-LLM

CapsFusion

[CVPR 2024] CapsFusion: Rethinking Image-Text Data at Scale

Spec-Bench

Spec-Bench: A Comprehensive Benchmark and Unified Evaluation Platform for Speculative Decoding (ACL 2024 Findings)

aligner

Achieving Efficient Alignment through Learned Correction

DiJiang

[ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear attention mechanism.

mixinglaws

linear_open_lm

A repository for research on medium sized language models.

HGRN

[NeurIPS 2023 spotlight] Official implementation of HGRN in our NeurIPS 2023 paper - Hierarchically Gated Recurrent Neural Network for Sequence Modeling

csl

[Preprint] Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Experts