conraddd

unitycatalog

Open, Multi-modal Catalog for Data & AI

openhouse

Open Control Plane for Tables in Data Lakehouse

kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

awesome-apache-airflow

Curated list of resources about Apache Airflow

spark-monitoring

Monitoring Azure Databricks jobs

kafka-delta-ingest

A highly efficient daemon for streaming data from Kafka into Delta Lake

pyre-check

Performant type-checking for python.

dbt-expectations

Port(ish) of Great Expectations to dbt test macros

delta-sharing

An open protocol for secure data sharing

podman-compose

a script to run docker-compose.yml using podman

podman

Podman: A tool for managing OCI containers and pods.

bamboolib

bamboolib - a GUI for pandas DataFrames

petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

delta-rs

A native Rust library for Delta Lake, with bindings into Python

dbx

🧱 Databricks CLI eXtensions - aka dbx is a CLI tool for development and advanced Databricks workflows management.

applied-ml

📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.

awesome-data-catalogs

📙 Awesome Data Catalogs and Observability Platforms.

jina

☁️ Build multimodal AI applications with cloud-native stack

delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

procfwk

A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

azure.datafactory.tools

Tools for deploying Data Factory (v2) in Microsoft Azure

files_in_repos

databricks-nutter-repos-demo

Demo of using the Nutter for testing of Databricks notebooks in the CI/CD pipeline

notebook-best-practices

An example showing how to apply software engineering best practices to Databricks notebooks.

azure-sdk-for-python

This repository is for active development of the Azure SDK for Python. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/python/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-python.

azure-pipelines-terraform

Azure Pipelines tasks for installing Terraform and running Terraform commands in a build or release pipeline.

pre-commit

A framework for managing and maintaining multi-language pre-commit hooks.

azure-keyvault-rotator

azure-pipelines-tasks

Tasks for Azure Pipelines

mlops-v2

Azure MLOps (v2) solution accelerators. Enterprise ready templates to deploy your machine learning models on the Azure Platform.