zarzen

distributed-training

openssl-example1

dt-autorun

autorun distributed training experiments and gathering logs

efa-bad-practice

pytorch

Tensors and Dynamic neural networks in Python with strong GPU acceleration

alpa

Training and serving large-scale neural networks

byteps

A high performance and generic framework for distributed DNN training

d2l-tvm

Dive into Deep Learning Compiler

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

DeepSpeedExamples

dlrm

An implementation of a deep learning recommendation model (DLRM)

doom.d

doom emacs config

efa-multithread

flash-attention

Fast and memory-efficient exact attention

grace

GRACE - GRAdient ComprEssion for distributed deep learning

horovod

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

horovod-examples

kickstart.nvim

A launch point for your personal nvim configuration

Megatron-LM

Ongoing research training transformer language models at scale, including: BERT & GPT-2

model-prepare

generate models to serve

nccl

adding timers for NCCL

nccl-fastsocket

NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.

nccl-tests

NCCL Tests

open_clip

An open source implementation of CLIP.

raf

ratex

Yuan's fork of Ratex

slapo

A schedule language for progressive optimization of large deep learning model training

split-annotations

Source code for the split annotations project.

UGATIT-pytorch

Official PyTorch implementation of U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

zarzen.github.io