stas00

porting

Helper scripts and notes that were used while porting various nlp models

gpt-neo-fine-tuning-example

Fine-Tune EleutherAI GPT-Neo And GPT-J-6B To Generate Netflix Movie Descriptions Using Hugginface And DeepSpeed

pytest-ipynb

A pytest plugin to run ipython/jupyter notebook cells as unit/integration tests

bash-tools

Bash tools

FlexFlow

A distributed deep learning framework that supports flexible parallelization strategies.

datasets

🤗 Fast, efficient, open-access datasets and evaluation metrics for Natural Language Processing and more in PyTorch, TensorFlow, NumPy and Pandas

apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

benchmarks

Benchmarks are the key to sustainable software developement

blog

Public repo for HF blog posts

code

Codebase for the project

DeepSpeedExamples

Example models using DeepSpeed

fairscale

PyTorch extensions for high performance and large scale training.

fairseq

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

fastai1

v1 of the fastai library. v2 is the current version. v1 is still supported for bug fixes, but will not receive new features.

functorch

functorch is a prototype of JAX-like composable function transforms for PyTorch.

machine-learning-note

my machine learning note

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Megatron-LM

Ongoing research training transformer models at scale

model_card

nestedtensor

[Prototype] Tools for the concurrent manipulation of variably sized Tensors.

nvidia-system-monitor-qt

Task Manager for Linux for Nvidia graphics cards

pytest-monitor

Pytest plugin for analyzing resource usage during test sessions

pytorch-tvmisc

Totally Versatile Miscellanea for Pytorch

pytorch_memlab

Profiling and inspecting memory in pytorch

tensorboard-tools

tensorboard tools

tokenizers

💥Fast State-of-the-Art Tokenizers optimized for Research and Production

tutorials

bitsandbytes

8-bit CUDA functions for PyTorch

make-tools

make tools

olm-datasets

Pipeline for pulling and processing online language model pretraining data from the web