leogao2

lm_dataformat

commoncrawl_downloader

toy-alignment-notebooks

The goal of this project is to collect beginner friendly notebooks for demonstrating alignment concepts in toy environments and make them feel more tangible.

gpt2-hf-to-tf1

Convert TF 2.x/Pytorch GPT2 model to original OpenAI TF 1.x GPT2 format

LLMD-CommonCrawl

apps

APPS: Automated Programming Progress Standard (NeurIPS 2021)

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

akronomicon

Public rankings of extreme-scale models

alignment-research-dataset

A dataset of alignment research and code to reproduce it

BIG-bench

Beyond the Imitation Game collaborative benchmark for enormous language models

cdict

CLIP

Contrastive Language-Image Pretraining

dask

Parallel computing with task scheduling

ergo

A Python library for integrating model-based and judgmental forecasting

gpt-neo

An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

GPT-Neo-visual-grounding

Visually ground GPT-Neo 1.3b and 2.7b

idemlib

learn2gradienthack

Uses learn2learn to train a fully connected network that implements the constant function and resists SGD

leogao2-site

link-randomizer

lm-evaluation-harness

mesh-transformer-jax

Model parallel transformers in JAX and Haiku

publish_perish

Academia sim

pythonic-rabbitmq

ray

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

research_helper

scaling-experiments

Experiments related to scaling laws for language models.

tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

The-Pile-FreeLaw

Download, parse, and filter data from Court Listener, part of the FreeLaw projects. Data-ready for The-Pile.

transformers

🤗 Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch.