data-catalog

There are 35 repositories under data-catalog topic.

• datahub

  datahub-project / datahub

  The Metadata Platform for your Data Stack

  data-catalogdata-discoverydatahubhacktoberfestlinkedinmetadata
  Language:Java 9299
• OpenMetadata

  open-metadata / OpenMetadata

  OpenMetadata is a unified platform for discovery, observability, and governance powered by a central metadata repository, in-depth lineage, and seamless team collaboration.

  metadatadatadiscoverydata-sciencedataqualitydata-profilingmetadata-managementdataengineeringdata-catalogdata-observabilitydbtdata-discoverydatacatalogdata-contractsdata-governancedata-lineagedata-validationsnowflakedata-qualitydata-quality-checksdata-collaboration
  Language:TypeScript 4308

• amundsen-io / amundsen

  Amundsen is a metadata driven application for improving the productivity of data analysts, data scientists and engineers when interacting with data.

  amundsendata-catalogdata-discoverylinuxfoundationmetadata
  Language:Python 4305
• odd-platform

  opendatadiscovery / odd-platform

  First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

  ossdata-platformmetadatametadata-managementdata-pipelinesdata-engineeringobservabilitydata-catalogdatacatalogdata-discoverydata-lineagebigdataalertinglineagedata-profilingdata-explorationdata-governancedata-qualitydata-sciencedata-observability
  Language:Java 1130

• intake / intake

  Intake is a lightweight package for finding, investigating, loading and disseminating data.

  pythondata-accessdata-catalog
  Language:Python 986

• rsyi / whale

  🐳 The stupidly simple CLI workspace for your data warehouse.

  data-catalogdata-discoverydata-documentation
  Language:Python 725
• awesome-data-catalogs

  opendatadiscovery / awesome-data-catalogs

  📙 Awesome Data Catalogs and Observability Platforms.

  data-catalogdata-discoverymetadatadataopsawesomeobservabilitydata-engineeringdata-qualitybig-dataopensourceopen-sourcemlawesome-listossopendatadatadiscoverymetadata-managementdatacatalog
  599

• datastrato / gravitino

  World's most powerful data catalog service with providing a high-performance, geo-distributed and federated metadata lake.

  ai-catalogdata-catalogdatalakefederated-querylakehousemetadatametalakemodel-catalogskycomputingstratosphere
  Language:Java 383
• recap

  recap-build / recap

  Work with your web service, database, and streaming schemas in a single format.

  data-catalogmetadatadata-discoveryrecapdata-engineeringdata-integrationdata-pipelinesetl
  Language:Python 306

• tokern / piicatcher

  Scan databases and data warehouses for PII data. Tag tables and columns in data catalogs like Amundsen and Datahub

  aws-athenaaws-gluepiidatabasedata-catalogphidatacatalogaws-redshiftsnowflakepython
  Language:Python 259
• meteor

  raystack / meteor

  Meteor is an easy-to-use, plugin-driven metadata collection framework to extract data from different sources and sink to any data catalog.

  metadatacollectorscraperextractorssinksbigdatadata-catalogdata-managementdataops
  Language:Go 171

• GoogleCloudPlatform / bigquery-data-lineage

  Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.

  data-lineagebigdatabigqueryzetasqldataflowdata-governancedata-managementdata-catalog
  Language:Java 137

• intake / intake-esm

  An intake plugin for parsing an Earth System Model (ESM) catalog and loading assets into xarray datasets.

  cmip6data-catalogpangeoearth-system-modeldata-accessintakeclimate-datasetscesm-lenshacktoberfest
  Language:Python 129

• getmetamapper / metamapper

  Metamapper is a data discovery and documentation platform for improving how teams understand and interact with their data.

  data-catalogdata-curationdata-discoverydata-warehousedjangometadatametamapperpythonschema-inspection
  Language:Python 76

• aws-samples / aws-dbs-refarch-datalake

  Reference Architectures for Datalakes on AWS

  data-lakedata-analyticsamazon-emringest-dataemr-clustergluehive-metastoredata-catalogdata-transformation
  Language:HTML 75

• GoogleCloudPlatform / datacatalog-connectors-rdbms

  Sample code with integration between Data Catalog and RDBMS data sources.

  pythonrdbmsdatabase-managementmetadata-managementmetadata-extractiongcpdatacatalogsqlserverpostgresqlmysqlteradataverticaredshiftgreenplumdata-catalogdatacatalog-connectors-rdbmsoracle
  Language:Python 72

• google / grizzly

  End-to-end DataOps platform deployed by Terraform.

  dataopsdataops-platformgcpterraformairflowbigquerycloud-storagecomposerdata-catalogdata-loss-preventiondataflowgitgoogle-cloudgoogle-cloud-platformpubsubspannercloud-sqldata-lineage
  Language:Python 56

• Tinkoff / data-detective

  Data catalog for everything in your company

  data-catalogmetadatadata-lineagedata-governancedata-detective
  Language:Python 48

• GoogleCloudPlatform / datacatalog-tag-engine

  Tag Engine automates the process of creating, updating, deleting, and populating metadata in bulk with Google Cloud's Data Catalog. Tag Engine is licensed under the Apache 2 license terms. Please make sure to read, understand and agree to the terms of the LICENSE and CONTRIBUTING files before proceeding.

  bigquerydata-catalogcloud-storagefirestorecloud-run
  Language:Python 46

• Bayer-Group / COLID-Documentation

  The documentation repository is part of the Corporate Linked Data Catalog - short: COLID - application.

  coliddata-cataloguedata-cataloglinked-datardfshaclfairfair-datafindableelasticsearchtriplestorecloud-native
  Language:HTML 43

• opendatadiscovery / odd-collector

  Open-source metadata collector based on ODD Specification

  data-catalogdata-discoverydata-governancedata-observabilitydata-platformdata-profilingdatacataloglineagedata-piplineshacktoberfest
  Language:Python 42

• ihsn / nada

  National Data Archive (NADA) is an open source data cataloging system that serves as a portal for researchers to browse, search, compare, apply for access, and download relevant census or survey information. It was originally developed to support the establishment of national survey data archives.

  data-catalogdata-catalog-managementdata-catalog-backenddata-portaldatacatalogcataloguenadadata-documentation-initiativeddi-codebook
  Language:PHP 37

• GoogleCloudPlatform / datacatalog-connectors-bi

  Sample code with integration between Data Catalog and BI data sources.

  pythongcpmetadata-managementmetadatadatacataloglookerlooker-sdktableauqlikqlik-sensedata-catalog
  Language:Python 32
• pace

  getstrm / pace

  Data policy IN, dynamic view OUT: PACE is the Policy As Code Engine. It helps you to programatically create and apply a data policy to a processing platform like Databricks, Snowflake or BigQuery (or plain 'ol Postgres, even!) with definitions imported from Collibra, Datahub, ODD and the like.

  data-catalogdata-processingpolicy-enforcementbigquerydata-contractsdata-governancedatabrickssnowflake
  Language:Kotlin 31

• commondataio / dataportals-registry

  Registry of data portals, catalogs, data repositories including data catalogs dataset and catalog description standard

  data-catalogdata-portaldata-repositorydatasetsregistrydata-discoveryopendatadatasetopen-data
  Language:Python 29
• stairlight

  tosh2230 / stairlight

  A data lineage tool detects table dependencies from rendered SQL statements.

  sqldata-lineageredashdbtbigquerydata-engineeringgcss3data-catalogdata-governancedata-opsdata-discoverydata-managementlineage
  Language:Python 26

• carte-data / carte

  A Python library to generate static data catalog sites. Carte scrapes metadata from your data assets and generates a fully searchable front end that's just HTML.

  data-catalogdata-discoverydata-documentationcartelightweight-data-catalogspython-library
  Language:Python 25

• awesome-mlops / awesome-data-management

  A curated list of awesome open source tools and commercial products to catalog, version, and manage data 🚀

  awesomeawesome-listdata-managementdataopsdataops-platformdataops-principlesdata-catalogdata-centric-aidata-lifecycledata-management-platformdata-managerdata-pipelinedata-sciencedata-toolsdata-workflowsdatascience
  22

• datopian / portal.js.bak

  🌀 The JS data presentation framework. For a single dataset to a full catalog.

  data-portal-frontendsdata-portaldata-catalogckandata-presentation
  Language:JavaScript 22

• odpi / egeria-docs

  Documentation repository for the Egeria project.

  egeriadocumentationmetadataopen-sourceopen-metadatametadata-managementmetadata-extractionmetadata-apimetadata-editormetadata-parserdata-catalogdata-cataloguedata-engineering
  Language:HTML 22

• apecs-org / Polar-EO-Database

  Polar Earth Observation Database of satellite sensors

  antarcticaarcticdata-catalogearth-observationhigh-mountain-asiaopen-scienceremote-sensingsatellite
  Language:Python 21

• dbt-content / google-datacatalog-dbt-tag

  Update a Google Data Catalog tag with dbt Cloud run metadata

  dbtgoogle-clouddata-catalogtaggoogle-datacatalogdbtcloud
  Language:Python 21

• SciCatProject / frontend

  SciCat open data catalogue web client

  data-catalogmetadata-catalogdata-managementdoi
  Language:TypeScript 21

• aaronspring / remote_climate_data

  a collection of remote climate data accessed via intake cached to disk

  data-catalogshapefilesaccessibilitynetcdfthredds-catalogsopendapobservationsclimate-scienceclimate-dataremote
  Language:Jupyter Notebook 19

• related-sciences / articat

  articat: data artifact catalog

  data-platformdata-discoverydata-managementdata-catalog
  Language:Python 16
• catalog_builder

  unytics / catalog_builder

  Data Catalogs Made Easy

  bigquerydata-catalogdata-discoverydatabricksdbtredshiftsnowflake
  Language:Python 15