data-lineage

There are 16 repositories under data-lineage topic.

• OpenMetadata

  open-metadata / OpenMetadata

  OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

  metadatadatadiscoverydata-sciencedataqualitydata-profilingmetadata-managementdataengineeringdata-catalogdata-observabilitydbtdata-discoverydatacatalogdata-contractsdata-governancedata-lineagedata-validationsnowflakedata-qualitydata-quality-checksdata-collaboration
  Language:TypeScript 5274
• elementary

  elementary-data / elementary

  The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

  data-lineagedata-governancedata-warehousesnowflakebigquerydata-analysisdata-pipelinesdata-pipelinelineagedata-reliabilitydata-observabilitydataopsdbtdbt-packagesanalytics-engineerdbt-artifactsredshift
  Language:HTML 1895

• MarquezProject / marquez

  Collect, aggregate, and visualize a data ecosystem's metadata

  data-dictionarydata-discoverydata-ecosystem-metadatadata-governancedata-lineagedata-opsdata-provenancemarquezmetadatametadata-service
  Language:Java 1741

• reata / sqllineage

  SQL Lineage Analysis Tool powered by Python

  data-discoverydata-governancedata-lineagelineagemetadatasql
  Language:Python 1297
• odd-platform

  opendatadiscovery / odd-platform

  First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

  ossdata-platformmetadatametadata-managementdata-pipelinesdata-engineeringobservabilitydata-catalogdatacatalogdata-discoverydata-lineagebigdataalertinglineagedata-profilingdata-explorationdata-governancedata-qualitydata-sciencedata-observability
  Language:Java 1204
• versatile-data-kit

  vmware / versatile-data-kit

  One framework to develop, deploy and operate data workflows with Python and SQL.

  data-sciencedata-engineeringsqltrinodata-lineageetleltdata-pipelinesdata-engineerdata-warehousewarehouseanalyticssnowflakedataopsdata-engineering-pipelinepythondatapipelinedata-structuresdatabase
  Language:Python 424

• elementary-data / dbt-data-reliability

  dbt package that is part of Elementary, the dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.

  datadbt-packagesdata-reliabilitydata-pipeline-monitoringdata-pipelinesanalyticsanalytics-engineeringdata-observabilitydata-lineagedbtdbt-artifactsdbt-tests
  Language:Python 376

• data-drift / data-drift

  Metrics Observability & Troubleshooting

  contextdata-monitoringdata-observabilitydata-qualitydbtmetricssemantic-layerdbt-metricsanalyticsdata-diffingdata-version-controldrill-downreconciliationbigquerydata-governancedata-lineagedata-reliabilitydbt-packagesredshiftsnowflake
  Language:HTML 318

• tokern / data-lineage

  Generate and Visualize Data Lineage from query history

  data-governancedata-lineagejupyterpostgresqlpython
  Language:Python 310

• grai-io / grai-core

  hacktoberfestdatadatalineagedbtopen-sourcepostgresqlpythonmssqlmysqlsnowflakedata-sciencedataengineeringfivetrandjangoparquetredshiftdata-lineage
  Language:Python 289

• finos / waltz

  Enterprise Information Service

  enterprise-softwarearchitecturesparkjavajooqangularjsenterprise-architectureenterprise-informationenterprise-toolsdata-lineageliquibase
  Language:Java 181

• tuva-health / tuva

  Main repo including core data model, data marts, reference data, terminology, and the clinical concept library

  analytics-engineeringbigquerydata-analyticsdata-governancedata-lineagedata-pipelinesdata-warehousedbtdbt-packageshealthcarehealthcare-analysishealthcare-dataopen-sourceredshiftsnowflakesqlterminology
  Language:Python 179

• GoogleCloudPlatform / bigquery-data-lineage

  Reference implementation for real-time Data Lineage tracking for BigQuery using Audit Logs, ZetaSQL and Dataflow.

  data-lineagebigdatabigqueryzetasqldataflowdata-governancedata-managementdata-catalog
  Language:Java 142

• slidoapp / dbt-superset-lineage

  Make dbt docs and Apache Superset talk to one another

  dbtsupersetdata-lineagelineageclitool
  Language:Python 133

• maropu / spark-sql-flow-plugin

  Visualize column-level data lineage in Spark SQL

  data-lineagegraphgraphvizneo4jpythonscalasparksqlvisualization
  Language:Scala 85

• thestyleofme / data-lineage-parent

  数据血缘，Hive/Sqoop/HBase/Spark等，发送到kafka后，解析处理使用neo4j生成血缘

  kafkaneo4jdata-lineagelineage-hooks
  Language:Java 78

• GitDataAI / jiaozifs

  A Git-like version control file system for data lineage & data collaboration.

  data-collaborationdata-versioningaiopsdata-meshdata-productdataopsdigital-twinsenterprise-datahubmlopsfederated-learningdata-lake-managementdata-version-controldata-lakedatalakegit-for-datagit-filesystemjiaozifsversion-controlled-filesystemdata-lineage
  Language:Go 73

• google / grizzly

  End-to-end DataOps platform deployed by Terraform.

  dataopsdataops-platformgcpterraformairflowbigquerycloud-storagecomposerdata-catalogdata-loss-preventiondataflowgitgoogle-cloudgoogle-cloud-platformpubsubspannercloud-sqldata-lineage
  Language:Python 62
• document-processing-pipeline-for-regulated-industries

  aws-samples / document-processing-pipeline-for-regulated-industries

  A boilerplate solution for processing image and PDF documents for regulated industries, with lineage and pipeline operations metadata services.

  machine-learningawscdkaws-lambdaamazon-web-servicesprocessing-pipelinesamazon-textractamazon-dynamodbamazon-elasticsearch-serviceamazon-comprehendamazon-s3amazon-snsamazon-sqsaws-cdkpdf-processingimage-processingdata-analyticsdata-lineageimage-processing-pythondata-governance
  Language:Python 61

• Tinkoff / data-detective

  Data catalog for everything in your company

  data-catalogmetadatadata-lineagedata-governancedata-detective
  Language:Python 50
• kun-scheduler

  miotech / kun-scheduler

  A workflow scheduler understands both your data and metadata.

  data-managementmetadatadata-lineage
  Language:Java 27
• stairlight

  tosh2230 / stairlight

  A data lineage tool detects table dependencies from rendered SQL statements.

  sqldata-lineageredashdbtbigquerydata-engineeringgcss3data-catalogdata-governancedata-opsdata-discoverydata-managementlineage
  Language:Python 27

• tomaztk / SQLServer-Data-Lineage

  Data Lineage for Microsoft SQL Server, Azure SQL Server and Azure Synapse

  data-lineagemicrosoft-sql-servert-sqlsqlsql-serversql-server-databaseremoveremove-commentsazure-sql-databaseazure-synapse
  Language:TSQL 17

• GuinsooLab / darkseal

  A Single place to Discover, Collaborate, and Get your data right

  metadatacatalogdata-catalogdata-discoverydata-platformdata-structuresmetadata-managementmetadata-standardguinsoolabdata-documentationdata-governancedata-lineagedata-securitydata-trustdata-collaborationdata-glossariesdata-compliancedata-alertingdata-notification
  Language:TypeScript 14

• tuva-health / tuva_demo

  A starter dbt project and synthetic claims dataset for trying out the Tuva Project.

  analytics-engineeringbigquerydata-analyticsdata-governancedata-lineagedata-pipelinesdata-warehousedbthealthcarehealthcare-analysishealthcare-datahealthcare-demoopen-sourceredshiftsnowflakesql
  14

• tuva-health / medicare_cclf_connector

  This connector is a dbt project that maps Medicare CCLF claims data to the Tuva Input Layer.

  analytics-engineeringbigquerydata-analyticsdata-governancedata-lineagedata-pipelinesdata-warehousedbthealthcarehealthcare-analysishealthcare-dataopen-sourceredshiftsnowflakesql
  12

• tuva-health / medicare_lds_connector

  Maps Medicare LDS claims data to the Tuva Input Layer so you can easily run the Tuva Project.

  analytics-engineeringbigquerydata-analyticsdata-governancedata-lineagedata-pipelinesdata-warehousedbthealthcarehealthcare-analysishealthcare-dataopen-sourceredshiftsnowflakesql
  10

• brunocampos01 / pyssas

  Build and deploy automated to SQL Server Analysis Services (SSAS) with Python.

  ssasssdtmeasurescalculated-columnscube-buildervisual-studiomicrosoft-sql-serveranalysis-servicesanalysis-services-tabularolapolap-cuberolapazure-ssasdata-lineagemeasure-dumpexport-sqlmssql-ssasssas-metadatassas-on-linuxpython
  Language:Python 9

• pi2schema / pi2schema

  Describe your Data Protection rules and Personal Identifying Information as part of your schema

  gdprgdpr-trackerlgpdschemaprotobufavrokafkaschema-registryschema-evolutiongovernancedata-governancedata-lineagecrypto-shreddinghacktoberfest
  Language:Java 9

• beingPeeDi / sqlsense

  Parse SQL statements and extract metadata and lineage information from it.

  data-lineagelineagesqlpostgresqlpostgrespythonparser
  Language:Python 8

• badoo / exasol-data-lineage

  Exasol data lineage scripts

  exasolexasol-dbluadata-lineage
  Language:Python 6

• IBM / multi-data-lineage-capture-py

  IBM Multi-Lineage Data System

  data-lineageartificial-intelligenceprovenancedata-sciencemachine-learning
  Language:Python 6

• tosh2230 / stairlight-app

  A web application rendering table dependency graph with tosh2230/stairlight, using Graphviz, Streamlit and Google Cloud Run.

  data-lineagecloudrunstreamlitgcpgraphviz
  Language:Python 5

• tuva-health / provider

  A dbt project that transforms messy public provider datasets into usable data for the Tuva Project.

  analytics-engineeringdata-analyticsdata-governancedata-lineagedata-pipelinesdata-warehousedbthealthcarehealthcare-analysishealthcare-dataopen-sourceproviderssnowflakesql
  3

• AbdullahMu / Data-Pipelines-with-Airflow

  Schedule, automate, and monitor data pipelines using Apache Airflow. Run data quality checks, track data lineage, and work with data pipelines in production.

  data-pipelinesamazon-redshiftairflowsqldata-warehouseapache-airflowdagetl-pipelines3-storagedata-quality-checksdata-lineage
  Language:Python 2

• Kuper-Tech / datahub-tags-distribution

  Airflow DAG for automated distribution of tags based on the Data Lineage from DataHub

  airflowdata-compliancedata-governancedata-lineagedatahubpii
  1