thedarkzeno

Open-Sora

Open-Sora: Democratizing Efficient Video Production for All

unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

pykan

Kolmogorov Arnold Networks

outlines

Structured Text Generation

llama-cpp-python

Python bindings for llama.cpp

LLMLingua

To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

efficient-kan

An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN).

matmulfreellm

Implementation for MatMul-free LM.

awesome-kan

A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and more, for researchers and developers in the Kolmogorov-Arnold Network field.

BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

LLMTest_NeedleInAHaystack

Doing simple retrieval from LLM models at various context lengths to measure accuracy

conditional-flow-matching

TorchCFM: a Conditional Flow Matching library

Triton-Puzzles

Puzzles for learning Triton

RETRO-pytorch

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

FourierKAN

attorch

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

quiet-star

Code for Quiet-STaR

minRF

Minimal implementation of scalable rectified flow transformers, based on SD3's approach

InfiniTransformer

Unofficial PyTorch/🤗Transformers(Gemma/Llama3) implementation of Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

fast-kan

FastKAN: Very Fast Implementation of Kolmogorov-Arnold Networks (KAN)

infini-transformer

PyTorch implementation of Infini-Transformer from "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" (https://arxiv.org/abs/2404.07143)

AutoCompressors

[EMNLP 2023] Adapting Language Models to Compress Long Contexts

BitMat

An efficent implementation of the method proposed in "The Era of 1-bit LLMs"

gym-chess

A simple chess environment for openai/gym

neuron_as_deep_net

Code behind the work "Single Cortical Neurons as Deep Artificial Neural Networks", published in Neuron 2021

smaller-transformers

Load What You Need: Smaller Multilingual Transformers for Pytorch and TensorFlow 2.0.

flow-matching

Annotated Flow Matching paper

1.58BitNet

Experimental BitNet Implementation

spacebyte

A byte-level decoder architecture that matches the performance of tokenized Transformers.

mamba2-torch