tianjianl

cartography

Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics

direct-preference-optimization

Reference implementation for DPO (Direct Preference Optimization)

pytorch-lightning

Pretrain, finetune and deploy AI models on multiple GPUs, TPUs with zero code changes.

ParroT

The ParroT framework to enhance and regulate the Translation Abilities during Chat based on open-sourced LLMs (e.g., LLaMA-7b, Bloomz-7b1-mt) and human written translation and evaluation data.

ALMA

State-of-the-art LLM-based translation models.

Tk-Instruct

Tk-Instruct is a Transformer model that is tuned to solve many NLP tasks by following instructions.

Glot500

Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023

TaiLr

ICLR2023 - Tailoring Language Generation Models under Total Variation Distance

RL4LMs

A modular RL library to fine-tune language models to human preferences

open-instruct

DITTO

The code of paper "Learning to Break the Loop: Analyzing and Mitigating Repetitions for Neural Text Generation" published at NeurIPS 2022

COMET

A Neural Framework for MT Evaluation

EKFAC-pytorch

Repository containing Pytorch code for EKFAC and K-FAC perconditioners.

LASER

Language-Agnostic SEntence Representations

llama-recipes

Scripts for fine-tuning Meta Llama with composable FSDP & PEFT methods to cover single/multi-node GPUs. Supports default & custom datasets for applications such as summarization and Q&A. Supporting a number of candid inference solutions such as HF TGI, VLLM for local or cloud deployment. Demo apps to showcase Meta Llama for WhatsApp & Messenger.

flores

Facebook Low Resource (FLoRes) MT Benchmark

mtdata

A tool that locates, downloads, and extracts machine translation corpora

llama

Inference code for Llama models

stopes

A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB team.

opus-100-corpus

awesome-RLHF

A curated list of reinforcement learning with human feedback resources (continually updated)

Megatron-LM

Ongoing research training transformer models at scale

gpt-neox

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

xmtf

Crosslingual Generalization through Multitask Finetuning

Intra-Distillation

This is the repository for our EMNLP 2022 paper "The Importance of Being Parameters: An Intra-Distillation Method for Serious Gains".

mesh-transformer-jax

Model parallel transformers in JAX and Haiku

pytorch-pruning

PyTorch Implementation of [1611.06440] Pruning Convolutional Neural Networks for Resource Efficient Inference

lm-evaluation-harness

A framework for few-shot evaluation of language models.

XLM

PyTorch original implementation of Cross-lingual Language Model Pretraining.

unify-parameter-efficient-tuning

Implementation of paper "Towards a Unified View of Parameter-Efficient Transfer Learning" (ICLR 2022)