data-testing

There are 1 repository under data-testing topic.

• soda-core

  sodadata / soda-core

  :zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io

  pythondata-engineeringdata-governancedata-monitoringdata-observabilitydata-profilingdata-qualitydata-quality-checksdata-quality-monitoringdata-reliabilitydata-testingdata-unit-testsdata-validationdataqualitydatatestingdbtpipeline-testingsnowflakedata-quality-testingdata-contracts
  Language:Python 2222

• re-data / re-data

  re_data - fix data issues before your users & CEO would discover them 😊

  data-monitoringdata-analysisdata-qualitydata-quality-monitoringopen-source-toolingdata-observabilitydataqualitydata-testingdata-quality-checksdbtdbt-packagesdata-reliability
  Language:HTML 1571
• piperider

  InfuseAI / piperider

  Code review for data in dbt

  data-pipelinedata-profilingdata-qualitydata-sciencedata-explorationedaexploratory-data-analysisdata-testingpythondata-observabilitydata-profilerdata-reliabilitydata-visualizationdbtdbt-metricscode-reviewreportingpull-requestscontinuous-integration
  Language:Python 492
• pointblank

  posit-dev / pointblank

  Data validation toolkit for assessing and monitoring data quality.

  data-qualitydata-testingdata-validationeasy-to-understandtabular-data
  Language:Python 296

• LukaszLapaj / software-testing-resource-pack

  Various files useful for manual testing and test automation etc.

  quality-assurancesoftware-testingsoftware-qualityresource-packtestingtesting-toolsui-teststest-dataresourcesfront-end-testingmanual-testingbootstrappingbackend-testingdata-testingtest-automatione2e-testingapi-testinge2e-tests
  201

• astronomer / airflow-provider-great-expectations

  Great Expectations Airflow operator

  data-scienceairflow-operatorsdata-qualitydata-testingairflowairflow-providers
  Language:Python 169

• re-data / dbt-re-data

  re_data - fix data issues before your users & CEO would discover them 😊

  data-monitoringdata-observabilitydata-qualitydata-testingdbtdbt-packagessql
  Language:Python 101

• akmalsoliev / Validoopsie

  A simple and easy to use Data Validation library for Python.

  pythondata-validationdata-engineeringdata-contractsdata-testing
  Language:Python 79
• data-caterer

  data-catering / data-caterer

  Test data management tool for any data source, batch or real-time. Generate, validate and clean up data all in one tool.

  data-generationdata-testingdata-validationtesting-automationdata-qualitytest-data-generatortest-data-managementautomated-test-generationdata-contractssynthetic-datadata-cleanup
  Language:Scala 71
• dataops-testgen

  DataKitchen / dataops-testgen

  DataOps Data Quality TestGen is part of DataKitchen's Open Source Data Observability. DataOps TestGen delivers simple, fast data quality test generation and execution by data profiling,  new dataset hygiene review, AI generation of data quality validation tests, ongoing testing of data refreshes, & continuous anomaly monitoring

  datadata-engineeringdata-observabilitydata-qualitydata-sciencedata-testingdatacheckerdataprofilingdataqualitydatavalidationself-hosteddataopsmssqlpostgresqlpythonredshiftsnowflake
  Language:Python 65

• sodadata / soda-spark

  Soda Spark is a PySpark library that helps you with testing your data in Spark Dataframes

  sparkpysparkdata-engineeringdata-qualitydata-observabilitydata-testingsoda-sqlpython
  Language:Python 64

• shridhar1504 / Sales-Forecasting-Datascience-Project

  Develop a data science project using historical sales data to build a regression model that accurately predicts future sales. Preprocess the dataset, conduct exploratory analysis, select relevant features, and employ regression algorithms for model development. Evaluate model performance, optimize hyperparameters, and provide actionable insights.

  data-analyticsdata-cleaningdata-sciencedata-testingdata-visualizationforecasting-modelsmachinmodel-evaluationmodel-fittingpredictionpredictive-modelingpython3regression-algorithmssklearn-librarysupervised-learningsalesforecast
  Language:Jupyter Notebook 17

• sodadata / soda-github-action

  :zap: Prevent downstream data quality issues by integrating the Soda Library into your CI/CD pipeline.

  data-engineeringdata-monitoringdata-observabilitydata-qualitydata-quality-checksdata-quality-monitoringdata-quality-testingdata-reliabilitydata-testingdata-unit-testsdata-validationdataqualitydatatestingpipeline-testingsnowflake
  Language:Python 14

• serialbandicoot / great-assertions

  This library is inspired by the Great Expectations library. The library has made the various expectations found in Great Expectations available when using the inbuilt python unittest assertions.

  testinggreat-expectationspython3databrickspythondata-sciencejupyter-notebookquality-assurancedata-testing
  Language:Python 10

• jafeerr / spark-data-test

  Spark Data Test - A PySpark-based automation testing utility to compare Spark DataFrames

  apache-sparkdata-testingdataframepyspark
  Language:Python 7
• data-caterer-example

  data-catering / data-caterer-example

  Example API implementation for Data Caterer

  batchdata-contractsdata-engineeringdata-gendata-generationdata-generatordata-qualitydata-testdata-testingdata-validationdata-validatorjavametadata-drivenreal-timescalatesting-automationyaml
  Language:Scala 6

• DataBridgeTech / dbqctl

  DataBridge Quality Control

  data-engineeringdata-profilingdata-qualitydata-quality-checksdata-quality-monitoringdataqualitydataqualitycheckdata-monitoringdata-observabilitydata-pipeline-monitoringdata-reliabilitydata-reliability-engineeringdata-testingdata-validationdata-validatordatatestingpipelines-testingclickhousemysqlpostgresql
  Language:Go 6

• andrjas / data_check

  data and pipeline testing with and for SQL

  data-engineeringdata-qualitydata-testingdata-unit-testsdatabasedataqualitypipelinesqldata-validation
  Language:Python 5

• neonexus / fixted

  Simple DB Fixtures for Sails.js v1 (fake data for testing).

  testingfixturesdatabase-testingfixtures-replacementsailssailsjsdata-testingmodel-testing
  Language:JavaScript 5

• pflooky / data-caterer

  Data generation and validation tool for any data source

  data-generationdata-validationdata-qualitydata-testing
  Language:Scala 5

• ericmjl / software-testing-open-source-and-data-science

  Software Testing in Open Source and Data Science: A talk delivered at the Data Umbrella speaker series

  data-sciencedata-testingmachine-learning-testingsoftware-testingtesting
  4

• pflooky / data-caterer-example

  Example API implementation for Data Caterer

  data-generationdata-generatordata-profilerdata-profilingdata-testingdata-validationdatageneratordockerdocker-composehelmjavakubernetesmetadatascalaschema-validationsqlsynthetic-data
  Language:Scala 3

• pflooky / data-caterer-docs

  Documentation for Data Caterer

  data-testingdata-validationdatagenerationdatageneratormetadatascalaschema-validationsynthetic-data
  Language:HTML 2

• manoj9788 / spark-etl-tests

  A sample repository showcasing, implementation of testing for ETL pipeline developed with Apache Spark

  scalaetletl-automationdata-testing
  1

• blleshi / Credit_Risk_Classification

  Credit Risk Classification

  classification-reportconfusion-matrixcredit-riskdata-testingimbalanced-learninglendingloanslogistic-regressionlogistic-regression-modelpandasrandomoversamplerresampled-datatarget-classificationcredit-risk-classificationdata-training
  Language:Jupyter Notebook 0

• ojasphansekar / Data-Management-Co-op

  National Grid ( Python, SQL Server, SSIS, SSRS, Tableau, Power BI, SQL Server Import Export Wizard, Data Validations, Data Integrations, Data Conversions )

  data-integrationdata-architecturedata-validationdata-analysis-pythondata-testingprocess-flow-diagramdata-modelingdata-mapping
  0

• RemoYukoff / aqueductus

  A data testing framework that executes queries on configurable data providers and validates the results with customizable YAML-defined assertions. Ensure data integrity, consistency, and reliability effortlessly.

  assertionsautomated-testingdata-completenessdata-integritydata-qualitydata-testingdata-validationpythonschema-validationunit-testing
  Language:Python 0

• afairless / kalman_filter

  Translating between two sets of notation for Kalman filters

  controlcontrol-systemsdata-testingfilterfilteringfilterskalman-filtermath-equationspyteststate-space-modelstate-space-modelsstatisticstime-seriestime-series-analysistime-series-forecasting
  Language:HTML

• Balajimohan18 / Sales-Forecasting-Datascience-Project

  Develop a data science project using historical sales data to build a regression model that accurately predicts future sales. Preprocess the dataset, conduct exploratory analysis, select relevant features, and employ regression algorithms for model development. Evaluate model performance, optimize hyperparameters, and provide actionable insights.

  data-analyticsdata-sciencedata-testingdata-visualizationforecastingforecasting-modelsmachine-learningmodel-evaluationpredictive-modelingpythonregression-algorithmssalesforecastscipysklearn-librarysupervised-learning
  Language:Jupyter Notebook

• JayLohokare / pySpark-data-testing-framework

  Dynamic data testing engine based on pySpark

  azuredata-testingdatabrickspysparktesting-framework
  Language:Jupyter Notebook

• neha-nayeem / machine-learning-challenge

  This project creates machine learning models capable of classifying candidate exoplanets from the raw dataset from NASA Kepler Space Telescope

  machine-learningsklearnjupyter-notebookpythonmodeltrainingmodel-trainingdata-testingpredictive-modeling

• siawayforward / dbt_about_it

  I'm learning how to use dbt with BigQuery so I can apply that knowledge wherever we end up working. It seems like a good DWH interface tool to know for data transformation and testing, and allows me to solidify concepts of testing in data ops.

  dbtdata-transformationdata-testing
  Language:Python