chuqbach

unitycatalog

Open, Multi-modal Catalog for Data & AI

polaris

Polaris Catalog is an open source catalog for Apache Iceberg

spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

ethereum-etl

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

ccxt

A JavaScript / TypeScript / Python / C# / PHP cryptocurrency trading API with support for more than 100 bitcoin/altcoin exchanges

xg2xg

by ex-googlers, for ex-googlers - a lookup table of similar tech & services

nessie

Nessie: Transactional Catalog for Data Lakes with Git-like semantics

localstack

💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline

data-contract-template

Template for a data contract used in a data mesh.

imessage-exporter

Export iMessage data + run iMessage Diagnostics

notes

Notes from tech books I'm reading.

ruff

An extremely fast Python linter and code formatter, written in Rust.

Data-Science-Extensions

winutils

winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows

git-changelog

Automatic Changelog generator using Jinja2 templates.

datahub

The Metadata Platform for your Data Stack

marquez

Collect, aggregate, and visualize a data ecosystem's metadata

OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

dvc

🦉 ML Experiments and Data Management with Git

piicatcher

Scan databases and data warehouses for PII data. Tag tables and columns in data catalogs like Amundsen and Datahub

data-lineage

Generate and Visualize Data Lineage from query history

sqllineage

SQL Lineage Analysis Tool powered by Python

avro-schema-viewer

Visualizer for Avro Schemas (.avsc) - Try it yourself at:

autometrics-rs

Easily add metrics to your code that actually help you spot and debug issues in production. Built on Prometheus and OpenTelemetry.

spark-xml

XML data source for Spark SQL and DataFrames

linearmouse

The mouse and trackpad utility for Mac.

TaskMatrix

meltano

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

autoscraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

roapi

Create full-fledged APIs for slowly moving datasets without writing a single line of code.