GuinsooLab / darkseal

A Single place to Discover, Collaborate, and Get your data right

Home Page:https://guinsoolab.github.io/glab/#/app/darkseal

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

badge
logo

Darkseal

Unlock the value of data assets with an end-to-end metadata platform that includes data discovery, governance, data quality, observability, and people collaboration.

Quickstart

wget https://raw.githubusercontent.com/GuinsooLab/darkseal/master/docker-compose.yml
docker-compose up -d

For more information, please referer to here.

Main Features

  • Data Trust
  • Data Documentation
  • Data Governance
  • Discovery & Collaboration
  • Teams & Users with Roles and Policies
  • Data Glossaries
  • Data Insights
  • Data Lineage
  • Security & Compliance
  • Alerts & Notifications

Core Components

Darkseal includes the following:

  • Metadata schemas - Defines core abstractions and vocabulary for metadata with schemas for Types, Entities, Relationships between entities. This is the foundation of the Open Metadata Standard.
  • Metadata store - Stores metadata graph that connects data assets, user, and tool generated metadata.
  • Metadata APIs - For producing and consuming metadata built on schemas for User Interfaces and Integration of tools, systems, and services.
  • Ingestion framework - A pluggable framework for integrating tools and ingesting metadata to the metadata store. Ingestion framework already supports well know data warehouses - Google BigQuery, Snowflake, Amazon Redshift, and Apache Hive, and databases - MySQL, Postgres, Oracle, MSSQL, and Guinsoo.
  • Metadata User Interface - One single place for users to discover, and collaborate on all data.

Snapshot & Gifs

Database Service

database_service

Workflow DAG

workflow_dag

Data Tables

data_tables

Table Columns

table_columns

Stream Lineage

stream_lineage

Documentation & Supports

Contributors

We ❤️ all contributions, big and small! Check out our CONTRIBUTING guide to get started and let us know how we can help.

Don't want to miss anything? Give the project a ⭐ 🚀

License

Darkseal is released under Apache License, Version 2.0

license

About

A Single place to Discover, Collaborate, and Get your data right

https://guinsoolab.github.io/glab/#/app/darkseal

License:Apache License 2.0


Languages

Language:TypeScript 41.8%Language:Java 27.8%Language:Python 24.7%Language:JavaScript 2.9%Language:Less 0.9%Language:CSS 0.8%Language:FreeMarker 0.4%Language:Shell 0.2%Language:HTML 0.2%Language:PLpgSQL 0.2%Language:Makefile 0.1%Language:Dockerfile 0.1%Language:ANTLR 0.0%Language:SCSS 0.0%Language:Jinja 0.0%Language:TSQL 0.0%