HyukjinKwon

congruity

The goal of this library is to provide a compatibility layer that makes it easier to adopt Spark Connect. The library is designed to be simply imported in your application and will then monkey-patch the existing API to provide the legacy functionality.

connectedhomeip

Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.

pyspark-ai

English SDK for Apache Spark

Nuitka

Nuitka is a Python compiler written in Python. It's fully compatible with Python 2.6, 2.7, 3.4-3.12. You feed it your Python app, it does a lot of clever things, and spits out an executable or extension module.

dolly

Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

jep

Embed Python in Java

py4j

Py4J enables Python programs to dynamically access arbitrary Java objects

dlt

Mirror of https://gitlab.com/zero323/dlt

honkit

:book: HonKit is building beautiful books using Markdown - Fork of GitBook

delta-sharing

An open protocol for secure data sharing

sphinx-plotly-directive

A directive for including a Plotly figure in a Sphinx document.

cluster-pack

A library on top of either pex or conda-pack to make your Python code easily available on a cluster

ApacheSparkGitHubActionImage

conda-pack

Package conda environments for redistribution

action-workflow_run-status

✅ The missing status check utility for workflow_run action.

spark-rapids

Spark RAPIDS plugin - accelerate Apache Spark with GPUs

pydata-sphinx-theme

A clean, three-column Sphinx theme with Bootstrap for the PyData community

spark-tpcds-datagen

All the things about TPC-DS in Apache Spark

pyspark-stubs

Apache (Py)Spark type annotations (stub files).

delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

pandas

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

koalas

Koalas: pandas API on Apache Spark

numba

NumPy aware dynamic Python compiler using LLVM

lljvm-translator

A lightweight library to inject LLVM bitcode into JVMs

jvm-profiler

JVM Profiler Sending Metrics to Kafka, Console Output or Custom Reporter

incubator-livy

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

spark-gotchas

Spark Gotchas. A subjective compilation of the Apache Spark tips and tricks

kubernetes

Production-Grade Container Scheduling and Management

awesome-consensus

Awesome list for Paxos and friends

awesome-json-datasets

A curated list of awesome JSON datasets that don't require authentication.