zhaohc10

alluxio

Alluxio, data orchestration for analytics and machine learning in the cloud

arrow-datafusion-comet

Apache Arrow DataFusion Comet Spark Accelerator

autoscaler

Autoscaling components for Kubernetes

batch-processing-gateway

The gateway component to make Spark on K8s much easier for Spark users.

beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

CloudShuffleService

Cloud Shuffle Service(CSS) is a general purpose remote shuffle solution for compute engines, including Spark/Flink/MapReduce.

cmkafka

distributed-compute-operator

:robot: Kubernetes operator providing Ray|Spark|Dask|MPI clusters on-demand

Firestorm

Firestorm is a Remote Shuffle Service, and provides the capability for Apache Spark and Apache Hadoop MapReduce applications to store shuffle data on remote servers

flink-remote-shuffle

Remote Shuffle Service for Flink

flyte

Kubernetes-native workflow automation platform for complex, mission-critical data and ML processes at scale. It has been battle-tested at Lyft, Spotify, Freenome, and others and is truly open-source.

foundationdb

FoundationDB - the open source, distributed, transactional key-value store

incubator-uniffle

Uniffle is a high performance, general purpose Remote Shuffle Service.

incubator-yunikorn-core

Apache YuniKorn Core

karpenter

Kubernetes Node Autoscaling: built for flexibility, performance, and simplicity.

kube-state-metrics

Add-on agent to generate and expose cluster-level metrics.

learning-k8s-source-code

近3年k8s源码分析笔记-1.17.4、client-go, docker, cni, etcd 读书笔记

linghu-algorithm-templete

nsinjector

Deploy resources to Kubernetes namespaces automatically

photon

The bucket of java back-end samples.

RemoteShuffleService

RemoteShuffleService-1

Remote shuffle service for Apache Spark to store shuffle data on remote servers.

servicetalk

A networking framework that evolves with your application

shuttle

Shuttle：High Available, High Performance Remote Shuffle Service

spark

Apache Spark

spark-s3-shuffle

A S3 Shuffle plugin for Apache Spark to enable elastic scaling for generic Spark workloads.

stocator

Stocator is high performing connector to object storage for Apache Spark, achieving performance by leveraging object storage semantics.

System-Design-1

System Design Bytes for Software Enthusiasts❤️. How is modern software designed? 🤔 Some design-level considerations for scalability, maintainability eventual consistency, availability & reliability. 👨‍💻 Interview Prep. 👨‍💻

the-algorithm

Source code for Twitter's Recommendation Algorithm

turicreate

Turi Create simplifies the development of custom machine learning models.