mmarius

peft

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

mesh-transformer-jax

Model parallel transformers in JAX and Haiku

FriendsDontLetFriends

Friends don't let friends make certain types of data visualization - What are they and why are they bad.

text-to-text-transfer-transformer

Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

x-transformers

A simple but complete full-attention transformer with a set of promising experimental features from various papers

OpenPrompt

An Open-Source Framework for Prompt-Learning.

SimCSE

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

adapters

A Unified Library for Parameter-Efficient and Modular Transfer Learning

makemore

An autoregressive character-level language model for making more things

pythia

The hub for EleutherAI's work on interpretability and learning dynamics

pyserini

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

cramming

Cramming the training of a (BERT-type) language model into limited compute.

bigscience

Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.

tueplots

Figure sizes, font sizes, fonts, and more configurations at minimal overhead. Fix your journal papers, conference proceedings, and other scientific publications.

mistral

Mistral: A strong, northwesterly wind: Framework for transparent and accessible large-scale language model training, built with Hugging Face 🤗 Transformers.

seqio

Task-based datasets, preprocessing, and evaluation for sequence models.

GradCache

Run Effective Large Batch Contrastive Learning Beyond GPU/TPU Memory Constraint

few-shot-learning

Few-shot Learning of GPT-3

glasbey

Algorithmically create or extend categorical colour palettes

Channel-LM-Prompting

An original implementation of "Noisy Channel Language Model Prompting for Few-Shot Text Classification"

GC-DPR

Train Dense Passage Retriever (DPR) with a single GPU

EditingClassifiers

architecture-objective

prompt_semantics

This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”

composable-sft

A library for parameter-efficient and composable transfer learning for NLP with sparse fine-tunings.

MCSE

NAACL 2022: MCSE: Multimodal Contrastive Learning of Sentence Embeddings

pet

This repository contains the code for "How many data points is a prompt worth?"

tilt-transfer

Code to run the TILT transfer learning experiments

bayesian-mi

This code accompanies the paper "Bayesian Framework for Information-Theoretic Probing" published in EMNLP 2021.