data-mesh

There are 10 repositories under data-mesh topic.

• datopian / datahub

  🌀 Rapidly build rich data portals using a modern frontend framework

  data-portaldata-portalsckandata-portal-frontendsdata-presentationdata-fabricdata-management-platformnextjsopen-data-portalreactdata-mesh
  Language:TypeScript 2171

• paypal / data-contract-template

  Template for a data contract used in a data mesh.

  datadata-contractdata-engineeringdata-mesh
  443

• merantix-momentum / squirrel-core

  A Python library that enables ML teams to share, load, and transform data in a collaborative, flexible, and efficient way :chestnut:

  pythonmlmachine-learningdata-sciencecomputer-visioncvnlpnatural-language-processingaipytorchtensorflowjaxdatasetsdistributeddataopsdeep-learningdata-meshdata-ingestioncloud-computingcollaboration
  Language:Python 279
• open-data-contract-standard

  bitol-io / open-data-contract-standard

  Home of the Open Data Contract Standard (ODCS).

  datadata-contractdata-contractsdata-engineeringdata-meshdata-quality
  211

• Azure / data-management-zone

  Template to deploy the Data Management Zone of Cloud Scale Analytics (former Enterprise-Scale Analytics). The Data Management Zone provides data governance and management capabilities for the data platform of an organization.

  enterprise-scalearmazurearchitecturepolicy-drivendata-platformbicepdata-meshdata-fabricenterprise-scale-analyticsdatamesh
  Language:Bicep 165

• Azure / data-landing-zone

  Template to deploy a single Data Landing Zone of the Data Management & Analytics Scenario (former Enterprise-Scale Analytics). The Data Landing Zone is a logical construct and a unit of scale in the architecture that enables data retention and execution of data workloads for generating insights and value with data.

  armazurearchitecturedata-platformenterprise-scalepolicy-drivenbicepdata-meshdata-fabricenterprise-scale-analyticsdatamesh
  Language:Bicep 159
• opendatadiscovery-specification

  opendatadiscovery / opendatadiscovery-specification

  ODD Specification is a universal open standard for collecting metadata.

  specificationspecopen-sourcebig-databig-data-platformdata-meshapidata-engineeringdata-discoveryopensourcedata-platformmetadatametadata-managementmetadata-parserdata-governance
  117

• Azure / data-product-analytics

  Template to deploy a Data Product for analytics and data science use-cases into a Data Landing Zone of the Data Management & Analytics Scenario (former Enterprise-Scale Analytics). The Data Product template can be used by cross-functional teams to create insights and products for external users.

  armazurearchitecturedata-platformenterprise-scalepolicy-drivendata-meshdata-fabricbicepenterprise-scale-analyticsdata-product
  Language:Bicep 72

• confluentinc / data-mesh-demo

  A Data Mesh proof-of-concept built on Confluent Cloud

  kafkadata-meshcloudconfluentconfluent-cloud
  Language:Java 62

• conduktor / conduktor-poc-kafka-protocol

  POC to demonstrate how to alter incoming/outgoing records in Kafka. It's a toy, don't use it in production.

  chaosdata-governancedata-meshkafkagateway
  Language:Java 59
• odm-specification-dpdescriptor

  opendatamesh-initiative / odm-specification-dpdescriptor

  The Data Product Descriptor Specification (DPDS) Repository

  data-contractdata-meshdata-productcomputational-governancedata-product-descriptordpds-specificationspecspecificationdata-fabricmetadata-management
  Language:HTML 57

• jupyter-naas / drivers

  Low-code Python library enabling access to APIs, tools, data sources in seconds.

  datadata-scienceetlintegrationlowcodeopen-sourceopensourcepipelinepipelinesdata-analysisdata-engineeringdata-integrationeltconnectorsdata-collectiondata-meshhacktoberfesthacktoberfest2022python
  Language:Python 53

• GitDataAI / jiaozifs

  An Git-like version control file system for data lineage & data collaboration.

  data-collaborationdata-versioningaiopsdata-meshdata-productdataopsdigital-twinsenterprise-datahubmlopsfederated-learningdata-lake-managementdata-version-controldata-lakedatalakegit-for-datagit-filesystemjiaozifsversion-controlled-filesystemdata-lineage
  Language:Go 45

• merantix-momentum / squirrel-datasets-core

  Squirrel dataset hub

  pythonnpldata-sciencemachine-learningnatural-language-processingaicomputer-visiondeep-learningtensorflowcvmlcollaborationpytorchdistributeddataopscloud-computingdatasetsdata-ingestiondata-mesh
  Language:Python 43

• Azure / data-product-batch

  Template to deploy a Data Product for Batch data processing into a Data Landing Zone of the Data Management & Analytics Scenario (former Enterprise-Scale Analytics). The Data Product template can be used by cross-functional teams to ingest, provide and create new data assets within the platform.

  armazurearchitecturedata-platformenterprise-scalepolicy-drivenbicepdata-meshdata-fabricenterprise-scale-analyticsdata-productdata-integration
  Language:Bicep 39

• Azure / data-product-streaming

  Template to deploy a Data Product for data stream processing into a Data Landing Zone of the Data Management & Analytics Scenario (former Enterprise-Scale Analytics). The Data Product template can be used by cross-functional teams to ingest, provide and create new data assets within the platform.

  armazurearchitecturedata-platformenterprise-scalepolicy-drivenbicepdata-meshdata-fabricenterprise-scale-analyticsdata-productdata-integration
  Language:Bicep 35
• data-platforms-tools

  ryandawsonuk / data-platforms-tools

  Guide to data platforms and tools

  azure-synapse-analyticsdata-warehousingbigquerydatabrickssnowflakedata-platformclouderadata-meshdata-governance
  31

• Mehdi-H / WeeklyCuration

  Interesting links I saw, every week

  architectureartificial-intelligencecloudcurationdata-meshddddevopsmlopsnewspythonveille-technique
  Language:Makefile 19

• O2-Czech-Republic / proxima-platform

  The Proxima platform.

  batch-processinganalytical-platformunified-data-processingapache-sparkapache-flinkapache-beamiot-platformdata-meshstream-processing
  Language:Java 18

• Iotic-Labs / iotics-identity-py

  Create Data Mesh. Use interoperable digital twins to create data interactions and build powerful real-time data products. This repository is a library for Decentralised Identity (DID) management with Iotics for applications in Python v3.8+.

  identitydidsdkpythonpython3ioticsdigital-twinsdata-interactiondata-meshidentity-apipypiidentities-managementbddiot
  Language:Python 16

• Iotic-Labs / iotics-identity-go

  Create Data Mesh. Use interoperable digital twins to create data interactions and build powerful real-time data products. This repository is a library for Decentralised Identity (DID) management with Iotics for applications in Golang.

  golangiotidentitysdkbdddididentity-apidata-interactiondata-meshdigital-twinsioticsidentities-management
  Language:Go 14

• agile-lab-dev / governance-decision-record

  The Governance Decision Record (GDR) is a specification model for (computational) data governance policies inspired from the ADR (Architectural Decision Record).

  architectural-decision-recordsdatadata-governancedata-managementdata-management-platformdata-meshplatformfederated-computational-governancegovernance-decision-recordpolicy-as-code
  Language:CUE 11
• learning-data-mesh

  lynnlangit / learning-data-mesh

  Repo with resources for learning Data Mesh

  data-meshdatamesh
  11

• findinpath / trino_data_mesh

  Proof of concept on how to gain insights with Trino across different databases from a distributed data mesh

  trinodbdata-meshdbt
  8

• opendatamesh-initiative / odm-platform

  A platform to manage the data product life cycle

  data-meshdata-managementdata-productdata-contractdata-fabriccomputational-governance
  Language:HTML 8
• ansilo

  ansilo-data / ansilo

  A Postgres interface into any database

  data-engineeringdata-meshdata-productsdata-virtualizationpostgresql-extensionsql-med
  Language:Rust 7

• Iotic-Labs / api

  Create Data Mesh. Use interoperable digital twins to create data interactions and build powerful real-time data products. This repository contains protocol buffers specification for Iotics API.

  digital-twinsprotobufprotobufdata-interactionioticsiotdata-mesh
  Language:Makefile 7

• EKGF / data-product

  Artifacts of the EKGF Data Product Workgroup

  data-meshdata-productekg
  Language:HTML 5

• limagbz / data-mesh-yelp

  A complete data product project used to study concepts related to data enginerring such as modeling, collection, operations and so on.

  data-engineeringkubernetespipelinesdata-meshgrafanagrafana-lokimicrok8sprometheusdevcontainerminioyelp-datasetchange-data-capturedebeziumkafkakafka-connect
  Language:Smarty 4

• ondat / charts

  Ondat Helm charts.

  helmchartshelm-chartsoperatorondatdata-meshpersistent-storagekubernetes
  Language:Shell 4

• ondat / documentation

  Ondat documentation content.

  ondatpersistent-storagetechnical-documentationmarkdowndocumentationdata-mesh
  Language:Shell 4

• findinpath / dbt_jaffle_shop_data_mesh

  A self-contained dbt project for showcasing the capabilities of the dbt-trino adapter

  dbtdbt-trinodata-mesh
  Language:Dockerfile 3
• riffl

  riffl / riffl

  Riffl - generic streaming data ingestion framework

  bigdataexactly-onceingestkafkaorcparquets3sinkstreaming-datarifflstreamingdata-lakedata-meshbig-datastream-processing
  Language:Java 2

• saba99 / Enhanced_3DHuman_Geneartion

  3D Human Generation With Mesh From 2D Images

  3d-generationdata-meshgenerative-aihuman-generationimage-to-3dmesh-generationmotion-generationavatar-generation
  Language:Jupyter Notebook 2

• mpearmain / data_contracts_sandbox

  This is a repo designed to show the workflow of a data contract, with pre-commit hooks and GitHub Actions on how to have the contract power a data platform

  data-contractsdata-engineering-infrastructuredata-mesh
  Language:Python 1

• thxnet / typescript-grpc-template

  A template for building TypeScript + gRPC service

  data-meshgrpcnodejsprotobuf3typescript
  Language:TypeScript 1