a3616001

llama

Inference code for LLaMA models

faiss

A library for efficient similarity search and clustering of dense vectors.

datasets

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

flash-attention

Fast and memory-efficient exact attention

triton

Development repository for the Triton language and compiler

DeepSeek-Coder

DeepSeek Coder: Let the Code Write Itself

annotated_latex_equations

Examples of how to create colorful, annotated equations in Latex using Tikz.

SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

UltraChat

Large-scale, Informative, and Diverse Multi-round Chat Data (and Models)

pyserini

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

state-spaces

Sequence Modeling with Structured State Spaces

performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

deduplicate-text-datasets

llama2.rs

A fast llama2 decoder in pure Rust.

RETRO-pytorch

Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

contriever

Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning

annotated-s4

Implementation of https://srush.github.io/annotated-s4

Awesome-LLM-System-Papers

nn_pruning

Prune a model while finetuning or training.

dpr-scale

Scalable training for dense retrieval models.

world-models

Extracting spatial and temporal world models from LLMs

CoFiPruning

[ACL 2022] Structured Pruning Learns Compact and Accurate Models https://arxiv.org/abs/2204.00408

rebias

Official Pytorch implementation of ReBias (Learning De-biased Representations with Biased Representations), ICML 2020

OptiPrompt

[NAACL 2021] Factual Probing Is [MASK]: Learning vs. Learning to Recall https://arxiv.org/abs/2104.05240

REST

REST: Retrieval-Based Speculative Decoding, NAACL 2024

EntityQuestions

EMNLP'2021: Simple Entity-centric Questions Challenge Dense Retrievers https://arxiv.org/abs/2109.08535

DinkyTrain

Princeton NLP's pre-training library based on fairseq with DeepSpeed kernel integration 🚃

performer

Simply Numpy implementation of the FAVOR+ attention mechanism, https://teddykoker.com/2020/11/performers/

kNN-KD

Code for paper "Nearest Neighbor Knowledge Distillation for Neural Machine Translation" by Zhixian Yang, Renliang Sun, and Xiaojun Wan. This paper is accepted by NAACL 2022 Main Conference.

rehashing

fast kernel evaluation in high dimensions via hashing