nccl

There are 0 repository under nccl topic.

cupy / cupy
NumPy & SciPy for GPU
cuda cudnn cublas cusolver nccl python numpy cupy curand cusparse gpu cutensor scipy nvtx nvrtc tensor cusparselt rocm
Language:Python 7847
huggingface / large_language_model_training_playbook
An open collection of implementation tips, tricks and resources for training large language models
cuda large-language-models llm nccl nlp performance python pytorch scalability troubleshooting
Language:Python 441
huggingface / llm_training_handbook
An open collection of methodologies to help with successful training of large language models.
cuda large-language-models llm nccl nlp performance python pytorch scalability troubleshooting
Language:Python 411
Bluefog-Lib / bluefog
Distributed and decentralized training framework for PyTorch over graph
mpi distributed-computing deeplearning machine-learning pytorch decentralized asynchronous one-sided nccl
Language:Python 290
microsoft / msrflute
Federated Learning Utilities and Tools for Experimentation
federated-learning distributed-learning gloo machine-learning nccl personalization privacy-tools pytorch simulation transformers-models
Language:Python 180
FZJ-JSC / tutorial-multi-gpu
Efficient Distributed GPU Programming for Exascale, an SC/ISC Tutorial
hpc mpi exascale-computing gpu multi-gpu sc21 supercomputing nccl nvshmem cuda isc22 sc22 sc23 isc23
Language:Cuda 144
google / nccl-fastsocket
NCCL Fast Socket is a transport layer plugin to improve NCCL collective communication performance on Google Cloud.
machine-learning nccl training
Language:C++ 102
JuliaGPU / NCCL.jl
A Julia wrapper for the NVIDIA Collective Communications Library.
julia cuda gpu nccl
Language:Julia 20
lanl / pyDNMFk
Python Distributed Non Negative Matrix Factorization with custom clustering
tensorfactorization nonnegative-matrix-factorization distributed-computing hpc mpi4py latent-features cupy machine-learning nccl outofmemory python
Language:Python 20
muriloboratto / NCCL
Sample examples of how to call collective operation functions on multi-GPU environments. A simple example of using broadcast, reduce, allGather, reduceScatter and sendRecv operations.
nccl cuda mpi laplacian-matrix
20
BaguaSys / bagua-net
High performance NCCL plugin for Bagua.
nccl bagua distributed-computing
Language:Rust 14
1duo / nccl-examples
NCCL Examples from Official NVIDIA NCCL Developer Guide.
nvidia nccl deep-learning distributed-systems
Language:CMake 11
openhackathons-org / nways_multi_gpu
N-Ways to Multi-GPU Programming
cuda hpc mpi nccl nsight-systems nvshmem
Language:C 9
YinLiu-91 / ncclOperationPlus
use ncclSend ncclRecv realize ncclSendrecv ncclGather ncclScatter ncclAlltoall
nccl cuda ncclsendrecv ncclgather ncclscatter ncclalltoall mpi cpp cplusplus
Language:Cuda 8
asprenger / distributed-training-patterns
Experiments with low level communication patterns that are useful for distributed training.
mpi mpi4py nccl horovod tensorflow distributed-training
Language:Python 5
UCBerkeley-Spring2022-CS267-project / blinkplus
Blink+: Increase GPU group bandwidth by utilizing across tenant NVLink.
nccl gpu collective-communication
Language:Jupyter Notebook 5
lancelee82 / pynccl
Nvidia NCCL2 Python bindings using ctypes and numba.
nccl numba ctypes python
Language:Python 4
dereklstinson / nccl
golang wrapper for nccl
cuda deep-learning go nccl parallel-computing
Language:Go 2
lancelee82 / necklace
Distributed deep learning framework based on pytorch/numba/nccl and zeromq.
distributed-deep-learning nccl zerorpc pytorch mxnet numba deep-learning distributed distributed-training
Language:Python 2
lcskrishna / nccl-rccl-parser
Tool to run rccl-tests/nccl-tests based on from an application and gather performance.
nccl rccl
Language:Python 2
muriloboratto / hands-on-supercomputing-with-parallel-computing
Hands-on Labs in Parallel Computing
cuda cuda-aware-mpi mpi nccl openmp
Language:Jupyter Notebook 2
radix-ai / scipy-notebook-gpu
jupyter/scipy-notebook with CUDA Toolkit, cuDNN, NCCL, and TensorRT
docker scipy-notebook tensorflow cuda cudnn nccl tensorrt
Language:Dockerfile 2
rodhuega / tfgMatrixNccl
Librería de operaciones matemáticas con matrices multi-gpu utilizando Nvidia NCCL.
nccl gpu-computing gpu gpu-programming cuda cpp11 nvidia
Language:Cuda 2
rohwid / auto-nvidia-cuda-driver
Installation script to install Nvidia driver and CUDA automatically in Ubuntu
nvidia-driver cuda cudnn nccl bash-scripting ubuntu1604 ubuntu1804
Language:Shell 2
melanie-t27 / 2024-EUMaster4HPC-Student-Challenge
EUMaster4HPC student challenge group 7 - EuroHPC Summit 2024 Antwerp
cg conjugate-gradient cuda efficiency mpi nccl openmp optimization parallel-programming scalability meluxina
Language:Cuda 0
MurrellGroup / Conflux.jl
Single-node data parallelism in Julia with CUDA
cuda data-parallelism flux julia nccl
Language:Julia 0
SquareFactory / ml-default
Default Docker image used to run experiments on csquare.run.
csquare cudnn deep-learning horovod isquare isquare-train machine-learning mxnet nccl pyspark python pytorch tensorflow torchvision train
Language:Dockerfile 0
GPU-Blood-Cell-Simulation / Simulation-Server
Blood Cell Simulation server
cuda hpc metaprogramming nccl opengl parallel-computing simulation
Language:C++
sub-mod / nccl-builds
nccl built on centos6
nccl

nccl

cupy / cupy

huggingface / large_language_model_training_playbook

huggingface / llm_training_handbook

Bluefog-Lib / bluefog

microsoft / msrflute

FZJ-JSC / tutorial-multi-gpu

google / nccl-fastsocket

JuliaGPU / NCCL.jl

lanl / pyDNMFk

muriloboratto / NCCL

BaguaSys / bagua-net

1duo / nccl-examples

openhackathons-org / nways_multi_gpu

YinLiu-91 / ncclOperationPlus

asprenger / distributed-training-patterns

UCBerkeley-Spring2022-CS267-project / blinkplus

lancelee82 / pynccl

dereklstinson / nccl

lancelee82 / necklace

lcskrishna / nccl-rccl-parser

muriloboratto / hands-on-supercomputing-with-parallel-computing

radix-ai / scipy-notebook-gpu

rodhuega / tfgMatrixNccl

rohwid / auto-nvidia-cuda-driver

melanie-t27 / 2024-EUMaster4HPC-Student-Challenge

MurrellGroup / Conflux.jl

SquareFactory / ml-default

GPU-Blood-Cell-Simulation / Simulation-Server

sub-mod / nccl-builds