yzhangcs

open_llama

OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

insanely-fast-whisper

lm-evaluation-harness

A framework for few-shot evaluation of language models.

IP-Adapter

The image prompt adapter is designed to enable a pretrained text-to-image diffusion model to generate images with image prompt.

distil-whisper

Distilled variant of Whisper for speech recognition. 6x faster, 50% smaller, within 1% word error rate.

dm-haiku

JAX-based neural network library

consistencydecoder

Consistency Distilled Diff VAE

kernl

Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.

cuda_programming

Code from the "CUDA Crash Course" YouTube series by CoffeeBeforeArch

LongChat

Official repository for LongChat and LongEval

gpt_paper_assistant

GPT4 based personalized ArXiv paper assistant bot

ALCE

[EMNLP 2023] Enabling Large Language Models to Generate Text with Citations. Paper: https://arxiv.org/abs/2305.14627

simplified_transformers

AutoCompressors

[EMNLP 2023] Adapting Language Models to Compress Long Contexts

flash-fft-conv

FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

DejaVu

aft-pytorch

Unofficial PyTorch implementation of Attention Free Transformer (AFT) layers by Apple Inc.

triton-transformer

Implementation of a Transformer, but completely in Triton

CoLT5-attention

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch

ModuleFormer

ModuleFormer is a MoE-based architecture that includes two different types of experts: stick-breaking attention heads and feedforward experts. We released a collection of ModuleFormer-based Language Models (MoLM) ranging in scale from 4 billion to 8 billion parameters.

REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024

Mirror

🪞A powerful toolkit for almost all the Information Extraction tasks.

latent-diffusion-for-language

heinsen_sequence

Code implementing "Efficient Parallelization of a Ubiquitious Sequential Computation" (Heinsen, 2023)

torchscale

Transformers at any scale

SeqBoat

[NeurIPS 2023] Sparse Modular Activation for Efficient Sequence Modeling

gated_linear_attention_layer

RAN

RAN: Recurrent Attention Networks for Long-text Modeling | Findings of ACL23

HGERE

Source Code for "Joint Entity and Relation Extraction with Span Pruning and Hypergraph Neural Networks"

fairseq-evo

Fairseq with transformer evolution