distributed-training

There are 12 repositories under distributed-training topic.

Made-With-ML
GokuMohandas / Made-With-ML
Learn how to design, develop, deploy and iterate on production-grade ML applications.
data-engineering data-quality data-science deep-learning distributed-ml distributed-training llms machine-learning mlops natural-language-processing python pytorch ray
Language:Jupyter Notebook 35635
huggingface / pytorch-image-models
PyTorch image models, scripts, pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNet-V3/V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
augmix convnext distributed-training dual-path-networks efficientnet image-classification imagenet maxvit mixnet mobile-deep-learning mobilenet-v2 mobilenetv3 nfnets normalization-free-training pretrained-models pretrained-weights pytorch randaugment resnet vision-transformer-models
Language:Python 29734
PaddlePaddle / Paddle
PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）
deep-learning distributed-training efficiency machine-learning neural-network paddlepaddle python scalability
Language:C++ 21606
PaddleNLP
PaddlePaddle / PaddleNLP
👑 Easy-to-use and powerful NLP and LLM library with 🤗 Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including 🗂Text Classification, 🔍 Neural Search, ❓ Question Answering, ℹ️ Information Extraction, 📄 Document Intelligence, 💌 Sentiment Analysis etc.
bert compression distributed-training document-intelligence embedding ernie information-extraction llama llm neural-search nlp paddlenlp pretrained-models question-answering search-engine semantic-analysis sentiment-analysis transformers uie
Language:Python 11407
skypilot-org / skypilot
SkyPilot: Run LLMs, AI, and Batch jobs on any cloud. Get maximum savings, highest GPU availability, and managed execution—all with a simple interface.
cloud-computing data-science deep-learning gpu hyperparameter-tuning machine-learning tpu job-queue job-scheduler cloud-management distributed-training ml-infrastructure multicloud spot-instances ml-platform cost-management cost-optimization finops llm-serving llm-training
Language:Python 5627
FedML-AI / FedML
FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, FEDML Nexus AI (https://fedml.ai) is your generative AI platform at scale.
ai-agent deep-learning distributed-training edge-ai federated-learning inference-engine machine-learning mlops model-deployment model-serving on-device-training
Language:Python 4058
Fengshenbang-LM
IDEA-CCNL / Fengshenbang-LM
Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。
chinese-nlp pretrained-models pytorch distributed-training transformers aigc multimodal
Language:Python 3879
bytedance / byteps
A high performance and generic framework for distributed DNN training
deep-learning distributed-training keras machine-learning mxnet pytorch tensorflow
Language:Python 3567
tensorflow / adanet
Fast and flexible AutoML with learning guarantees.
automl deep-learning distributed-training ensemble gpu learning-theory machine-learning neural-architecture-search python tensorflow tpu
Language:Jupyter Notebook 3471
alpa-projects / alpa
Training and serving large-scale neural networks with auto parallelization.
deep-learning machine-learning compiler distributed-training high-performance-computing alpa jax distributed-computing llm auto-parallelization
Language:Python 2983
determined
determined-ai / determined
Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.
deep-learning machine-learning ml-platform ml-infrastructure hyperparameter-optimization hyperparameter-search distributed-training pytorch tensorflow hyperparameter-tuning kubernetes data-science mlops keras
Language:Go 2855
learning-at-home / hivemind
Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.
asynchronous-programming asyncio deep-learning dht distributed-systems distributed-training hivemind machine-learning mixture-of-experts neural-networks pytorch volunteer-computing
Language:Python 1836
tensorlayer / HyperPose
Library for Fast and Flexible Human Pose Estimation
tensorlayer tensorflow openpose pose-estimation computer-vision distributed-training tensorrt mobilenet neural-networks
Language:Python 1238
DeepRec-AI / DeepRec
DeepRec is a high-performance recommendation deep learning framework based on TensorFlow. It is hosted in incubation in LF AI & Data Foundation.
deep-learning machine-learning python scalability distributed-training recommendation-engine search-engine advertising
Language:C++ 967
intelligent-machine-learning / dlrover
DLRover: An Automatic Distributed Deep Learning System
distributed-training k8s llm-training
Language:Python 939
mryab / efficient-dl-systems
Efficient Deep Learning Systems course materials (HSE, YSDA)
deep-learning efficient-deep-learning pytorch cuda distributed-training machine-learning ml-infrastructure mlops
Language:Jupyter Notebook 577
alibaba / Megatron-LLaMA
Best practice for training LLaMA models in Megatron-LM
deepspeed distributed-training llama llm megatron-lm pretraining pytorch
Language:Python 530
adaptdl
petuum / adaptdl
Resource-adaptive cluster scheduler for deep learning training.
deep-learning kubernetes pytorch distributed-systems aws distributed-training machine-learning cloud
Language:Python 401
Guitaricet / relora
Official code for ReLoRA from the paper Stack More Layers Differently: High-Rank Training Through Low-Rank Updates
deep-learning distributed-training llama nlp peft transformer
Language:Jupyter Notebook 398
Oneflow-Inc / libai
LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
oneflow nlp deep-learning large-scale data-parallelism model-parallelism distributed-training pipeline-parallelism transformer self-supervised-learning vision-transformer
Language:Python 376
DataCanvasIO / HyperGBM
A full pipeline AutoML tool for tabular data
automl gbm fullpipeline xgboost lightgbm catboost pseudo-labeling adversarial-validation semi-supervised-learning datacleaning preprocessing ensemble-learning tabular-data distributed-training dask gpu-acceleration dask-distributed rapidsai sklearn
Language:Python 322
pytorch / torchx
TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.
pytorch machine-learning kubernetes slurm distributed-training pipelines components deep-learning python aws-batch ray airflow
Language:Python 296
lsds / KungFu
Fast and Adaptive Distributed Machine Learning for TensorFlow, PyTorch and MindSpore.
tensorflow keras distributed-training distributed-systems
Language:Go 285
HandyRL
DeNA / HandyRL
HandyRL is a handy and simple framework based on Python and PyTorch for distributed reinforcement learning that is applicable to your own environments.
reinforcement-learning pytorch games policy-gradient deep-learning machine-learning distributed-training
Language:Python 282
maudzung / YOLO3D-YOLOv4-PyTorch
YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud (ECCV 2018)
yolov4 darknet point-cloud real-time yolo3d rotated-boxes-iou 3d-object-detection object-detection distributed-training
Language:Python 282
awslabs / deeplearning-cfn
Distributed Deep Learning on AWS Using CloudFormation (CFN), MXNet and TensorFlow
deeplearning tensorflow mxnet ec2-instance aws aws-cloudformation distributed-training deep-learning-ami
Language:Python 256
alibaba / EasyParallelLibrary
Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
deep-learning data-parallelism model-parallelism pipeline-parallelism memory-efficient distributed-training gpu
Language:Python 252
nanodl
HMUNACHI / nanodl
A Jax-based library for designing and training transformer models from scratch.
attention-mechanism deep-learning distributed-training gpt jax llama machine-learning nlp transformer attention flax mistral
Language:Python 251
dougsouza / pytorch-sync-batchnorm-example
How to use Cross Replica / Synchronized Batchnorm in Pytorch
pytorch batchnorm distributed-training dataparallel
247
PKU-DAIR / Hetu
A high-performance distributed deep learning system targeting large-scale and automated distributed training.
deep-learning distributed-systems distributed-training high-dimensional machine-learning gpu autograd data-science artificial-intelligence python deep-neural-networks embeddings state-of-the-art
Language:Python 231
synxlin / deep-gradient-compression
[ICLR 2018] Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
deep-learning distributed-training deep-gradient-compression gradient-compression sparse-distributed-training
Language:Python 203
deepglint / unicom
universal visual model trained on LAION-400M
large-sacle-pretrained-model universal-image-retrieval iclr2023 laion400m vision-transformer distributed-training retrieval-anything in-shop
Language:Python 195
wenwei202 / terngrad
Ternary Gradients to Reduce Communication in Distributed Deep Learning (TensorFlow)
distributed-training sgd quantization deep-learning deep-neural-networks data-parallelism
Language:Python 181
ZJU-OpenKS / OpenKS
OpenKS - 领域可泛化的知识学习与计算引擎
knowledge-computing kgqa distributed-training
Language:Python 153
PaddlePaddle / PLSC
Paddle Large Scale Classification Tools，supports ArcFace, CosFace, PartialFC, Data Parallel + Model Parallel. Model includes ResNet, ViT, Swin, DeiT, CaiT, FaceViT, MoCo, MAE, ConvMAE, CAE.
face-recognition arcface cosface partial-fc data-parallel model-parallel large-scale paddlepaddle paddle distributed-training hight-speed deit resnet vit facevit cait moco-v3 mae convmae swin-transformer
Language:Python 144
huggingface / chug
Minimal sharded dataset loaders, decoders, and utils for multi-modal document, image, and text datasets.
computer-vision dataloading datasets distributed-training document-understanding multi-modal-learning pdf-document webdataset
Language:Python 130