Kyriection

stochastorch

A Pytorch implementation of stochastic addition.

LLM-Adapters

Code for our EMNLP 2023 Paper: "LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models"

FourierKAN

pykan

Kolmogorov Arnold Networks

infini-transformer-pytorch

Implementation of Infini-Transformer in Pytorch

LLM-FP4

The official implementation of the EMNLP 2023 paper LLM-FP4

PiPPy

Pipeline Parallelism for PyTorch

searchformer

Official codebase for the paper "Beyond A* Better Planning with Transformers via Search Dynamics Bootstrapping".

flash-linear-attention

Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton

SnapKV

CUDA_gemm

A simple high performance CUDA GEMM implementation.

Usage-of-the-8bit-Quantization-in-Neural-Network-Training

This repo has the script to reproduce the experiments in project 'Usage of the 8bit Quantization in Neural Network Training'.

attorch

A subset of PyTorch's neural network modules, written in Python using OpenAI's Triton.

llama3

The official Meta Llama 3 GitHub site

qlora

QLoRA: Efficient Finetuning of Quantized LLMs

megalodon

Reference implementation of Megalodon 7B model

improved-t5

Experiments for efforts to train a new and improved t5

Q-Hitter

CUDA-Learn-Notes

🎉 Modern CUDA Learn Notes with PyTorch: fp32, fp16, bf16, fp8/int8, flash_attn, sgemm, sgemv, warp/block reduce, dot, elementwise, softmax, layernorm, rmsnorm.

mistral-inference

Official inference library for Mistral models

SVD-LLM

Official Code for "SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression"

recurrentgemma

Open weights language model from Google DeepMind, based on Griffin.

llm.c

LLM training in simple, raw C/CUDA

VAR

[GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

pyreft

ReFT: Representation Finetuning for Language Models

schedule_free

Schedule-Free Optimization in PyTorch

GRIFFIN

JetMoE

Reaching LLaMA2 Performance with 0.1M Dollars

SWE-agent

SWE-agent takes a GitHub issue and tries to automatically fix it, using GPT-4, or your LM of choice. It can also be employed for offensive cybersecurity or competitive coding challenges.

dbrx

Code examples and resources for DBRX, a large language model developed by Databricks