RuanVisser

tpunicorn

Babysit your preemptible TPUs

faiss

A library for efficient similarity search and clustering of dense vectors.

cleanlab

The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

electra

ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

cramming

Cramming the training of a (BERT-type) language model into limited compute.

jailbreak_llms

[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

peS2o

Pretraining Efficiently on S2ORC!

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

BERT-related-papers

BERT-related papers

socialreaper

Social media scraping / data collection library for Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs

fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

COCO-LM

[NeurIPS 2021] COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining

metro_t0

Code repo for "Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers" (ACL 2023)

wtpsplit

Toolkit to segment text into sentences or other semantic units in a robust, efficient and adaptable way.

fzf

:cherry_blossom: A command-line fuzzy finder

cheat.sh

the only cheat sheet you need

BiBERT

This is the repository of the EMNLP 2021 paper "BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translation".

a-pretrainers-guide

guidance

A guidance language for controlling large language models.

Indic-BERT-v1

Indic-BERT-v1: BERT-based Multilingual Model for 11 Indic Languages and Indian-English. For latest Indic-BERT v2, check: https://github.com/AI4Bharat/IndicBERT

ltg-bert

LTG-Bert

vega

A visualization grammar.

JARVIS

JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf

transformer-alignment

Code for EMNLP 2020 paper Accurate Word Alignment Induction from Neural Machine Translation

the-algorithm

Source code for Twitter's Recommendation Algorithm

the-algorithm-ml

Source code for Twitter's Recommendation Algorithm

FinGPT

FinGPT: Open-Source Financial Large Language Models! Revolutionize 🔥 We release the trained model on HuggingFace.

massive

Tools and Modeling Code for the MASSIVE dataset

DeBERTa

The implementation of DeBERTa

KoBERT

Korean BERT pre-trained cased (KoBERT)