CZWin32768

XNLG

AAAI-20 paper: Cross-Lingual Natural Language Generation via Pre-Training

XLM-Align

bitsandbytes-aarch64

aarch64 for bitsandbytes

CCLUE

古文语言理解测评基准 Classical Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

unilm

UniLM AI - Unified "Language" Model Pre-training across Tasks, Languages, and Modalities

accelerate

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

adapter-transformers

Huggingface Transformers + Adapters = ❤️

alpaca-lora

Instruct-tune LLaMA on consumer hardware

awesome-chatgpt-prompts

This repo includes ChatGPT prompt curation to use ChatGPT better.

cloudflare-scrape

A Python module to bypass Cloudflare's anti-bot page.

crosslingual_winograd

"It's All in the Heads" (Findings of ACL 2021), official implementation and data

DeepSpeedExamples

Example models using DeepSpeed

DialoGPT

Large-scale pretraining for dialogue

dockers

flores

Facebook Low Resource (FLoRes) MT Benchmark

improved-aesthetic-predictor

CLIP+MLP Aesthetic Score Predictor

llama.cpp

Port of Facebook's LLaMA model in C/C++

MAGIC

Language Models Can See: Plugging Visual Controls in Text Generation

massive

Tools and Modeling Code for the MASSIVE dataset

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Neural-Collapse

[NeurIPS 2021] A Geometric Analysis of Neural Collapse with Unconstrained Features

open_clip

An open source implementation of CLIP.

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

ParlAI

A framework for training and evaluating AI models on a variety of openly available dialogue datasets.

PrefixTuning

Prefix-Tuning: Optimizing Continuous Prompts for Generation

RL4LMs

A modular RL library to fine-tune language models to human preferences

RWKV-LM

RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.

transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

trlx

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

YaLM-100B

Pretrained language model with 100B parameters