yzhangcs

datasets

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

Bend

A massively parallel, high-level programming language

llama3-from-scratch

llama3 implementation one matrix multiplication at a time

matmulfreellm

Implementation for MatMul-free LM.

Phi-3CookBook

This is a Phi-3 book for getting started with Phi-3. Phi-3, a family of open AI models developed by Microsoft. Phi-3 models are the most capable and cost-effective small language models (SLMs) available, outperforming models of the same size and next size up across a variety of language, reasoning, coding, and math benchmarks.

DeepSeek-Coder-V2

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

xlstm

Official repository of the xLSTM.

streaming

A Data Streaming Library for Efficient Neural Network Training

attention-cnn

Source code for "On the Relationship between Self-Attention and Convolutional Layers"

gemma-2B-10M

Gemma 2B with 10M context length using Infini-attention.

StreamSpeech

StreamSpeech is an “All in One” seamless model for offline and simultaneous speech recognition, speech translation and speech synthesis.

MAP-NEO

Samba

Official implementation of "Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling"

magvit2-pytorch

Implementation of MagViT2 Tokenizer in Pytorch

MS-AMP

Microsoft Automatic Mixed Precision Library

Agent-Attention

Official repository of Agent Attention (ECCV2024)

cosmopedia

InfLLM

The code of our paper "InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory"

transformer-sequential

Trains Transformer model variants. Data isn't shuffled between batches.

seqax

seqax = sequence modeling + JAX

LCKV

Layer-Condensed KV cache w/ 10 times larger batch size, fewer params and less computation. Dramatic speed up with better task performance. Accepted to ACL 2024.

block-transformer

Block Transformer: Global-to-Local Language Modeling for Fast Inference (Official Code)

triton-index

Cataloging released Triton kernels.

RWKV-PEFT

uncheatable_eval

Evaluating LLMs with Dynamic Data

MoE-SFT

🍼 Official implementation of Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts

UnifiedImplicitAttnRepr

hypernetwork-attention

Official code for the paper "Attention as a Hypernetwork"

Dep_Transformer_Grammars

GL-DancingMen

Cipher font of "The Adventure of the Dancing Men", The Return of Sherlock Holmes by Arthur Conan Doyle.