fsdp

There are 0 repository under fsdp topic.

SohamGovande / podplex
🦾💻🌐 distributed training & serverless inference at scale on RunPod
decentralized distributed fsdp inference llm runpod serverless
Language:Jupyter Notebook 17
GURPREETKAURJETHRA / Meta-LLAMA3-GenAI-UseCases-End-To-End-Implementation-Guides
META LLAMA3 GENAI Real World UseCases End To End Implementation Guide
chromadb fine-tuning fsdp generativeai langchain-python llama3 llama3-70b-8192 llama3-finetune llama3-meta-ai llama3-prompts llama3-rag prompt-tuning pytorch qlora rag sagemaker streamlit huggingface ollama
Language:Jupyter Notebook 14
AlibabaPAI / FlashModels
Fast and easy distributed model training examples.
distributed-training xla data-parallelism deep-learning fsdp model-parallelism pytorch sequence-parallelism tensor-parallelism zero llm
Language:Python 9
arawxx / FSDP-Distributed-Training-of-ConvNextV2-on-CIFAR10
A script for training the ConvNextV2 on CIFAR10 dataset using the FSDP technique for a distributed training scheme.
convnext convnextv2 deep-learning distributed distributed-computing distributed-learning distributed-training fsdp fully-sharded-data-parallel pytorch
Language:Python 7
SulRash / minLLMTrain
Minimal yet high performant code for pretraining llms. Attempts to implement some SOTA features. Implements training through: Deepspeed, Megatron-LM, and FSDP. WIP
deepspeed fsdp huggingface llm megatron-lm pretraining
Language:Python 5
abhilash1910 / Framework-Optimization
Framework, Model & Kernel Optimizations for Distributed Deep Learning - Data Hack Summit
codegen ddp deepspeed fsdp inductor pytorch triton pipelineparallel tensorparallel
Language:Python 1
ridwan-salau / transformer-xl
Fully Sharded Data Parallel (FSDP) implementation of Transformer XL
fsdp fully-sharded-data-parallel pytorch transformer
Language:Python 0