HazyResearch

ThunderKittens

Tile primitives for speedy kernels

data-centric-ai

Resources for Data Centric AI

safari

Convolutions for Sequence Modeling

meerkat

Creative interactive views of any dataset.

hgcn

Hyperbolic Graph Convolutional Networks in PyTorch.

hyena-dna

Official implementation for HyenaDNA, a long-range genomic foundation model built with Hyena

m2

Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"

evaporate

This repo contains data and code for the paper "Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes"

manifest

Prompt programming with FMs.

aisys-building-blocks

Building blocks for foundation models.

legalbench

An open science effort to benchmark legal reasoning in foundation models

flash-fft-conv

FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

based

Code for exploring Based models from "Simple linear attention language models balance the recall-throughput tradeoff"

lolcats

Repo for "LoLCATs: On Low-Rank Linearizing of Large Language Models"

hippo-code

butterfly

Butterfly matrix multiplication in PyTorch

zoology

Understand and test language model architectures on synthetic tasks.

domino

eclair-agents

Automating enterprise workflows with multimodal agents

structured-nets

Structured matrices for compressing neural networks

train-tk

train with kittens!

prefix-linear-attention

skill-it

Skill-It! A Data-Driven Skills Framework for Understanding and Training Language Models

wonderbread

WONDERBREAD benchmark + dataset for BPM tasks

aioli

Aioli: A unified optimization framework for language model data mixing

based-evaluation-harness

A framework for few-shot evaluation of language models.

olive-evaluation-harness

A framework for few-shot evaluation of language models.

axolive

Go ahead and axolotl questions

olive-recipes

Scripts for fine-tuning Meta Llama3 with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama3 for WhatsApp & Messenger.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs