dataquality
There are 13 repositories under dataquality topic.
great-expectations / great_expectations
Always know what to expect from your data.
Language:Python 9863cleanlab / cleanlab
The standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
Language:Python 9446- OpenMetadata
open-metadata / OpenMetadata
OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
Language:TypeScript 5275 awslabs / deequ
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
Language:Scala 3264- Language:Python 2935
- soda-core
sodadata / soda-core
:zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io
Language:Python 1873 - Language:HTML 1543
- zingg
zinggAI / zingg
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
Language:Java 950 - chaos_genius
chaos-genius / chaos_genius
ML powered analytics engine for outlier detection and root cause analysis.
Language:Python 728 datacleaner / DataCleaner
The premier open source Data Quality solution
Language:Java 591datavane / datavines
Know your data better！Datavines is Next-gen Data Observability Platform, support metadata manage and data quality.
Language:Java 447- Language:Python 329
- Language:Python 158
- dcs-core
datachecks / dcs-core
Open Source Data Quality Monitoring.
Language:Python 137 AutoViML / pandas_dq
Find data quality issues and clean your data in a single line of code with a Scikit-Learn compatible Transformer.
Language:Python 124- Language:JavaScript 121
- data-observability-installer
DataKitchen / data-observability-installer
Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team is the first to know and the first to solve with visibility across and down your data estate. Save time with simple, fast data quality test generation and execution. Trust your data, tools, and systems end to end.
Language:Python 76 - Language:Java 43
infinitelambda / dq-tools
Make simple storing test results and visualisation of these in a BI dashboard
Language:PLpgSQL 36- dataops-testgen
DataOps Data Quality TestGen is part of DataKitchen's Open Source Data Observability. DataOps TestGen delivers simple, fast data quality test generation and execution by data profiling, new dataset hygiene review, AI generation of data quality validation tests, ongoing testing of data refreshes, & continuous anomaly monitoring
Language:Python 35 AltimateAI / datapilot-cli
Datailot-cli is the command line interface for accessing the AI teammate for engineers to ensure best practices in their SQL and dbt projects.
Language:Python 21- Language:C++ 17
BirdiD / BirdiDQ
BirdiDQ leverages the power of the Python Great Expectations open-source library and combines it with the simplicity of natural language queries to effortlessly identify and report data quality issues, all at the tip of your fingers.
Language:Jupyter Notebook 14Open Standard for Metadata. A Single place to Discover, Collaborate and Get your data right.
Language:CSS 14bikash / DataQuality
Tutorial and examples of Data Quality in Big Data System
grillazz / fastapi-greatexpectations
Run greatexpectations.io on ANY SQL Engine using REST API. Supported by FastAPI, Pydantic and SQLAlchemy as best data quality tool
Language:Python 12HuemulSolutions / huemul-bigdatagovernance
Huemul BigDataGovernance, es una framework que trabaja sobre Spark, Hive y HDFS. Permite la implementación de una estrategia corporativa de dato único, basada en buenas prácticas de Gobierno de Datos. Permite implementar tablas con control de Primary Key y Foreing Key al insertar y actualizar datos utilizando la librería, Validación de nulos, largos de textos, máximos/mínimos de números y fechas, valores únicos y valores por default. También permite clasificar los campos en aplicabilidad de derechos ARCO para facilitar la implementación de leyes de protección de datos tipo GDPR, identificar los niveles de seguridad y si se está aplicando algún tipo de encriptación. Adicionalmente permite agregar reglas de validación más complejas sobre la misma tabla.
Language:Scala 11:zap: Prevent downstream data quality issues by integrating the Soda Library into your CI/CD pipeline.
Language:Python 11josephmachado / data-quality-w-greatexpectations
Code for data quality with greatexpectations blog
Language:Python 10- openclients
Data-Culpa / openclients
Open source clients for working with Data Culpa Validator services from data pipelines
Language:Python 9 - Language:Java 9
ydataai / ydata-talkdatatome
Make your dataset talk to you. The AI assistant for data preparation.
Language:Python 9devoteamgcloud / dataform-assertions
Enhance your data testing seamlessly with this Dataform package, featuring robust common assertions to ensure the accuracy and integrity of your warehouse data.
Language:JavaScript 8- setup-duckdb-action
opt-nc / setup-duckdb-action
🦆 Blazing Fast and highly customizable Github Action to setup a DuckDb runtime
Language:JavaScript 8 - Language:Python 8
- SQL-DQC
martandsingh / SQL-DQC
SQL based data profiling & data quality checks, which will help you to perform data profiling & data quality checks on SQL database at table & database level.
Language:TSQL 7
