Rohan Varma's repositories
rohan-blog
Software Engineer @ Facebook
torchtune-2
A Native-PyTorch Library for LLM Fine-tuning
ao
The torchao repository contains api's and workflows for quantization and pruning gpu models.
d2go
D2Go is a toolkit for efficient deep learning
DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
fairscale
PyTorch extensions for high performance and large scale training.
gloo
Collective communications library with various primitives for multi-machine training.
llama-recipes
Examples and recipes for Llama model
metaseq
Repo for external large-scale work
mixprecision
modular repo to work on bert attn issue with mixed precision
multimodal
TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.
t5_11
housing our model example of fine tuning an 11B t5 with FSDP
tnt
A lightweight library for PyTorch training tools and utilities
torchdistx
Torch Distributed Experimental
torchrec
Pytorch domain library for recommendation systems
torchrec-1
Pytorch domain library for recommendation systems
torchtune
A Native-PyTorch Library for LLM Fine-tuning
transformer_framework
framework for plug and play of various transformers (vision and nlp) with FSDP
transformer_nuggets
A place to store reusable transformer components of my own creation or found on the interwebs
tutorials
PyTorch tutorials.
xla
Enabling PyTorch on XLA Devices (e.g. Google TPU)