stas00

ipyexperiments

jupyter/ipython experiment containers for GPU and general RAM re-use

porting

Helper scripts and notes that were used while porting various nlp models

ml-ways

ML/DL Math and Method notes

transformers

🤗Transformers: State-of-the-art Natural Language Processing for Pytorch and TensorFlow 2.0.

git-tools

helper git tools

toolbox

Various tools in my toolbox

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

FlexFlow

A distributed deep learning framework that supports flexible parallelization strategies.

pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

datasets

🤗 Fast, efficient, open-access datasets and evaluation metrics for Natural Language Processing and more in PyTorch, TensorFlow, NumPy and Pandas

stas00

accelerate

A simple way to train and use NLP models with multi-GPU, TPU, mixed-precision

apex

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

blog

Public repo for HF blog posts

code

Codebase for the project

DeepSpeedExamples

Example models using DeepSpeed

fairscale

PyTorch extensions for high performance and large scale training.

fastai1

v1 of the fastai library. v2 is the current version. v1 is still supported for bug fixes, but will not receive new features.

functorch

functorch is a prototype of JAX-like composable function transforms for PyTorch.

gpt-neo-fine-tuning-example

Fine-Tune EleutherAI GPT-Neo And GPT-J-6B To Generate Netflix Movie Descriptions Using Hugginface And DeepSpeed

machine-learning-note

my machine learning note

Megatron-DeepSpeed

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Megatron-LM

Ongoing research training transformer models at scale

nvidia-system-monitor-qt

Task Manager for Linux for Nvidia graphics cards

pytest-monitor

Pytest plugin for analyzing resource usage during test sessions

pytorch-tvmisc

Totally Versatile Miscellanea for Pytorch

pytorch_memlab

Profiling and inspecting memory in pytorch

test-attribution

tokenizers

💥Fast State-of-the-Art Tokenizers optimized for Research and Production

tutorials