datacleaner

There are 3 repositories under datacleaner topic.

• great-expectations / great_expectations

  Always know what to expect from your data.

  cleandatadata-engineeringdata-profilersdata-profilingdata-qualitydata-sciencedata-unit-testsdatacleanerdatacleaningdataqualitydataunittestedaexploratory-analysisexploratory-data-analysisexploratorydataanalysismlopspipelinepipeline-debtpipeline-testingpipeline-tests
  Language:Python 9540

• datacleaner / DataCleaner

  The premier open source Data Quality solution

  datadata-analysisdata-sciencedatabasedatacleanerdataqualitydesktopetlmdmprofiling
  Language:Java 571
• data-observability-installer

  DataKitchen / data-observability-installer

  Installer for DataKitchen's Open Source Data Observability Products. Data breaks. Servers break. Your toolchain breaks. Ensure your team is the first to know and the first to solve with visibility across and down your data estate. Save time with simple, fast data quality test generation and execution. Trust your data, tools, and systems end to end.

  datadata-engineeringdata-observabilitydata-profilingdata-qualitydata-sciencedatacleanerdatacleaningdataopsdataqualitymssqlpipeline-testspostgresqlredshiftself-hostedsnowflakedatacheckerdatatestingdatavalidationdata-reliability
  Language:Python 41

• catalyst / moodle-local_datacleaner

  Reduce, filter, and anonymize moodle data for non-prod environments

  anonymizedata-cleaningdatacleanermoodlephpplugin
  Language:PHP 19

• Nelson-Gon / mde

  mde: Missing Data Explorer

  data-analysisdata-cleaningdata-explorationdata-sciencedatacleanerdatacleaningexploratory-data-analysismissingmissing-datamissing-value-treatmentmissing-valuesmissingnessomitrr-packager-statsrecodereplacerstatsstatistics
  Language:R 4

• Ashishkr007 / CSVParser

  CSVParser is a tool to parse csv file using univocity and commons csv parsers. It cleans new line (\n) character & special characters between data. It also handle various garbage data like odd no of quotes or delimiters in side quotes. It validate each record with specified delimiter count and separate it out to _GoodRecords.CSV and _BadRecords.CSV file. This is a Data Cleaner tool to run before ingestion to Data Lake. It make sure data is in right csv format to build table on it.

  univocitycsv-parseropencsvdatacleanernewlinequotesgarbage-segregationdatacleaningdatacleansingcsvparser
  Language:Java 0

• Ashishkr007 / Data-Cleaner

  If you have some csv file and having CRLF, LF in between data and you want to create some table (Hive table). You will face issue that some of column have null value. It’s because line terminator in hive is \n and if and \n or \r coming between data it treating as line terminator before actual line terminator and rest for column is getting null value. I tried multiple option like spark, hive serde and many more but I found good with perl. Today I a sharing my Perl script to remove all newline and special characters.

  newlinenewline-character-sequencesdatacleaningdatacleanercrlfperl-scriptcsv-parser
  Language:Perl 0

• WxFang / Recipe-Recommendation

  Get data from Yummly API. Utilize nutrition theory as filter of recipes . Find similar flavor based on unsupervised clustering. Finally build up website to recoomend recipes based on flavor and nuitrition.

  flaskclusteringpythondatacleaner
  Language:Python 0