distributed-computing

There are 81 repositories under distributed-computing topic.

ColossalAI
hpcaitech / ColossalAI
Making large AI models cheaper, faster and more accessible
ai big-model data-parallelism deep-learning distributed-computing foundation-models heterogeneous-training hpc inference large-scale model-parallelism pipeline-parallelism
Language:Python 38011
ty4z2008 / Qix
Machine Learning、Deep Learning、PostgreSQL、Distributed System、Node.Js、Golang
machine-learning distributed-systems postgresql deep-learning go awesome distributed-computing distributed-database
14542
hazelcast / hazelcast
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
java hazelcast in-memory big-data scalability distributed caching hacktoberfest stream-processing low-latency distributed-computing distributed-systems data-in-motion data-insights real-time
Language:Java 5909
protoactor-go
asynkron / protoactor-go
Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
grpc actor-model actors cross-platform protobuf akka clustering distributed-computing go distributed-systems golang
Language:Go 4891
chrislusf / gleam
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
distributed-computing distributed-systems golang map-reduce
Language:Go 3381
catalyst-team / catalyst
Accelerated deep learning R&D
deep-learning reinforcement-learning machine-learning computer-vision pytorch python distributed-computing infrastructure research reproducibility image-processing image-classification image-segmentation object-detection natural-language-processing text-classification text-segmentation information-retrieval recommender-system metric-learning
Language:Python 3233
alpa-projects / alpa
Training and serving large-scale neural networks with auto parallelization.
deep-learning machine-learning compiler distributed-training high-performance-computing alpa jax distributed-computing llm auto-parallelization
Language:Python 2989
jostmey / NakedTensor
Bare bone examples of machine learning in TensorFlow
big-data distributed-computing linear-regression simple tensorflow tensorflow-examples tensorflow-exercises tensorflow-tutorials
Language:Python 2423
heidihoward / distributed-consensus-reading-list
A list of papers about distributed consensus.
distributed-systems paxos atomic-broadcast-protocol zookeeper fault-tolerance distributed-computing consensus-algorithms quorum-systems state-machine-replication consistency replication
2292
microsoft / GraphEngine
Microsoft Graph Engine
distributed-computing dotnet graph-engine graph-query-language in-memory-computations in-memory-storage
Language:C# 2182
BOINC / boinc
Open-source software for volunteer computing and grid computing.
boinc distributed-computing volunteer-computing high-performance-computing citizen-science scientific-computing grid-computing science high-throughput-computing php c-plus-plus hacktoberfest
Language:PHP 1926
rl
pytorch / rl
A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.
ai control decision-making distributed-computing machine-learning marl model-based-reinforcement-learning multi-agent-reinforcement-learning pytorch reinforcement-learning rl robotics torch
Language:Python 1921
fugue-project / fugue
A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
dask data-practitioners distributed distributed-computing distributed-systems machine-learning pandas spark sql
Language:Python 1887
Daft
Eventual-Inc / Daft
Distributed DataFrame for Python designed for the cloud, powered by Rust
machine-learning python data-engineering data-science dataframe distributed-computing rust big-data
Language:Rust 1751
asynkron / protoactor-dotnet
Proto Actor - Ultra fast distributed actors for Go, C# and Java/Kotlin
actors clustering proto-actor akka distributed-systems distributed-computing
Language:C# 1669
innovation-cat / Awesome-Federated-Machine-Learning
Everything about federated learning, including research papers, books, codes, tutorials, videos and beyond
federated-learning machine-learning deep-learning privacy-preserving-machine-learning computer-vision distributed-computing edge-computing security differential-privacy
1593
moosefs / moosefs
MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)
software-defined-storage posix filesystem file-system distributed-file-system clustering distributed-storage distributed-computing fuse big-data snapshot storage-tiering high-availability scalability storage moosefs hadoop posix-compliant petabyte ditributed-systems
Language:C 1592
maxpumperla / elephas
Distributed Deep learning with Keras & Spark
spark keras neural-networks deep-learning distributed-computing
Language:Python 1572
dask / distributed
A distributed task scheduler for Dask
pydata dask distributed-computing python hacktoberfest
Language:Python 1544
wx-chevalier / DistributedSystem-Notes
:books: 深入浅出分布式基础架构，Linux 与操作系统篇 | 分布式系统篇 | 分布式计算篇 | 数据库篇 | 网络篇 | 虚拟化与编排篇 | 大数据与云计算篇
operatingsystem distributed-computing distributed-systems maintainance storage virtualization network
Language:HTML 1502
eclipse-zenoh / zenoh
zenoh unifies data in motion, data in-use, data at rest and computations. It carefully blends traditional pub/sub with geo-distributed storages, queries and computations, while retaining a level of time and space efficiency that is well beyond any of the mainstream stacks.
networking protocol messaging storage edge-computing iot robotics embedded ros2 zenoh rust distributed-computing distributed-storage distributed-systems geo-distributed-storages network-programming rust-lang
Language:Rust 1271
AshwinRJ / Federated-Learning-PyTorch
Implementation of Communication-Efficient Learning of Deep Networks from Decentralized Data
federated-learning distributed-computing deep-learning pytorch python
Language:Python 1193
TsingZ0 / PFLlib
We expose this user-friendly algorithm library (with an integrated evaluation platform) for beginners who intend to start federated learning (FL) study
non-iid federated-learning personalization pytorch python distributed-computing heterogeneity differential-privacy privacy imagenet domainnet iot attack dlg privacy-protection
Language:Python 1179
google / vizier
Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service.
vizier hyperparameter-optimization tuning tuning-parameters blackbox-optimization hyperparameter-tuning bayesian-optimization evolutionary-algorithms distributed-systems distributed-computing grpc open-source google algorithm deep-learning machine-learning optimization
Language:Python 1176
lensacom / sparkit-learn
PySpark + Scikit-learn = Sparkit-learn
scikit-learn apache-spark machine-learning distributed-computing python
Language:Python 1148
holochain / holochain
The current, performant & industrial strength version of Holochain on Rust.
holochain holochain-rust p2p blockchain dapps-development distributed-computing dapps rust web3
Language:Rust 1119
mahmoudparsian / data-algorithms-book
MapReduce, Spark, Java, and Scala for Data Algorithms Book
hadoop-mapreduce java distributed-computing scala mapreduce data-algorithms python machine-learning pyspark distributed-algorithms mappers reducers apache-hadoop apache-spark design-patterns partitioning
Language:Java 1061
SmartSql
dotnetcore / SmartSql
SmartSql = MyBatis in C# + .NET Core+ Cache(Memory | Redis) + R/W Splitting + PropertyChangedTrack +Dynamic Repository + InvokeSync + Diagnostics
orm dotnet-core cross-platform high-performance distributed-computing redis zookeeper read-write-separation cache sql smartsql dotnet dotnetcore mybatis sqlserver mysql sqlite postgresql skywalking oracle
Language:C# 1051
goflow
s8sg / goflow
A Golang based high performance, scalable and distributed workflow framework
workflow-engine workload-automation framework golang workflow distributed-computing
Language:CSS 1044
fiber
uber / fiber
Distributed Computing for AI Made Simple
python distributed-computing multiprocessing sandbox machine-learning
Language:Python 1039
Haidra-Org / AI-Horde
A crowdsourced distributed cluster for AI art and text generation
ai distributed-computing flask-api gpt opt stable-diffusion volunteer-computing
Language:Python 994
lizardfs / lizardfs
LizardFS is an Open Source Distributed File System licensed under GPLv3.
c-plus-plus gplv3 nas macosx linux posix distributed-systems distributed-computing fault-tolerance high-performance high-availability qos erasure-coding replication replicas geo-replication hsm hierarchical-storage lizardfs software-defined-storage
Language:C++ 946
future
HenrikBengtsson / future
:rocket: R package: future: Unified Parallel and Distributed Processing in R for Everyone
r cran parallel-processing parallel-computing distributed-computing hpc-clusters hpc promises futures asynchronous programming parallelization
Language:R 934
couler-proj / couler
Unified Interface for Constructing and Managing Workflows on different workflow engines, such as Argo Workflows, Tekton Pipelines, and Apache Airflow.
cloud-native workflow-management machine-learning distributed-computing kubernetes workflow-engine argo-workflows tekton-pipelines apache-airflow unified-interface unified-api scheduler workflow-automation kubeflow python
Language:Python 891
BaguaSys / bagua
Bagua Speeds up PyTorch
distributed pytorch rust-lang deep-learning distributed-systems distributed-computing machine-learning bagua
Language:Python 865
gojek / awesome-distributed-systems
Awesome list of distributed systems resources
awesome awesome-list distributed-systems distributed distributed-database distributed-computing distributed-storage computer-science computer-engineering
799