tianjianl

pytorch-lightning

Pretrain, finetune and deploy AI models on multiple GPUs, TPUs with zero code changes.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

trl

Train transformer language models with reinforcement learning.

OpenNMT-py

Open Source Neural Machine Translation and (Large) Language Models in PyTorch

pythia

The hub for EleutherAI's work on interpretability and learning dynamics

RL4LMs

A modular RL library to fine-tune language models to human preferences

direct-preference-optimization

Reference implementation for DPO (Direct Preference Optimization)

TransformerLens

A library for mechanistic interpretability of GPT-style language models

P-tuning

A novel method to tune language models. Codes and datasets for paper ``GPT understands, too''.

vec2text

utilities for decoding deep representations (like sentence embeddings) back to text

sam

dont-stop-pretraining

Code associated with the Don't Stop Pretraining ACL 2020 paper

ALMA

State-of-the-art LLM-based translation models.

BRIO

ACL 2022: BRIO: Bringing Order to Abstractive Summarization

pytorch_influence_functions

This is a PyTorch reimplementation of Influence Functions from the ICML2017 best paper: Understanding Black-box Predictions via Influence Functions by Pang Wei Koh and Percy Liang.

doremi

Pytorch implementation of DoReMi, a method for optimizing the data mixture weights in language modeling datasets

dsir

DSIR large-scale data selection framework for language model training

cartography

Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics

Tk-Instruct

Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.

ParroT

The ParroT framework to enhance and regulate the Translation Abilities during Chat based on open-sourced LLMs (e.g., LLaMA-7b, Bloomz-7b1-mt) and human written translation and evaluation data.

wimbd

What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets

QuRating

[ICML 2024] Selecting High-Quality Data for Training Language Models

Glot500

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023

submodlib

Summarize Massive Datasets using Submodular Optimization

emergent_in_context_learning

Gradient_Starvation

Gradient Starvation: A Learning Proclivity in Neural Networks

TaiLr

ICLR2023 - Tailoring Language Generation Models under Total Variation Distance

InstructMT

A collection of instruction data and scripts for machine translation.

ParetoMNMT

Source code for paper "On the Pareto Front of Multilingual Neural Machine Translation" @ NeurIPS 2023

Integer_Addition

✱ Understanding the underlying learning dynamics of simple tasks in Transformer networks