gspandy / datavines

Know your data better！Datavines is Next-gen Data Observability Platform, support metadata manage and data quality.

DataVines

---

Data quality is used to ensure the accuracy of data in the process of integration and processing. It is also the core component of DataOps. DataVines is an easy-to-use data quality service platform that supports multiple metric.

Architecture Design

Install

Need: Maven 3.6.1 and later

$ mvn clean package -Prelease -DskipTests

Features of DataVines

• Easy to use

• Built in multiple Metric、ExpectedType、ResultFormula

  • Metric Plugins
  • ExpectedType Plugins
  • ResultFormula Plugins

• Modular and plug-in mechanism, easy to extend

  • Engine Plugins
  • Connector Plugins
  • Register Plugins
  • Notification Plugins

• Support Spark 2.x、JDBC Engine

Environmental dependency

1. java runtime environment: jdk8
2. If the data volume is small, or the goal is merely for functional verification, you can use JDBC engine
3. If you want to run DataVines based on Spark, you need to ensure that your server has spark installed

Quick start

QuickStart

Development

Developer Guide

Contribution

For guides on how to contribute, visit: Contribution Guidelines

RoadMap

V1.0.0 RoadMap

Contact Us

[e-mail]

License

DataVines is licensed under the Apache License 2.0. DataVines relies on some third-party components, and their open source protocols are also Apache License 2.0 or compatible with Apache License 2.0. In addition, DataVines also directly references or modifies some codes in Apache DolphinScheduler, SeaTunnel and Dubbo, all of which are Apache License 2.0. Thanks for contributions to these projects.