Huanglk's repositories
transpeeder
train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism
bigscience
Central place for the engineering/scaling WG: documentation, SLURM scripts and logs, compute environment and data.
cs224n-winter-2017
All lecture notes, slides and assignments from CS224n: Natural Language Processing with Deep Learning class by Stanford
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.
lightning-transformers
Flexible components pairing 🤗 Transformers with Pytorch Lightning
nebullvm
Plug and play modules to optimize the performances of your AI systems 🚀
Open-Assistant
OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.
trlx
A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)