datawarehouse

There are 24 repositories under datawarehouse topic.

• dinky

  DataLinkDC / dinky

  Dinky is a real-time data development platform based on Apache Flink, enabling agile data development, deployment and operation.

  flinkflinksqlreal-time-computing-platformflinkcdcolapsqldatalakedatawarehouse
  Language:Java 3607

• hydradatabase / columnar

  Postgres-native columnar storage extension

  data-warehousedatawarehousepostgrespostgresqlpostgresql-extension
  Language:C 3002

• getdozer / dozer

  Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.

  apisdatarustsqlrealtimestreamingetllow-codepostgressnowflakeapiclickhousedatawarehousedebe
  Language:Rust 1566

• simbafl / DataWarehouse

  从数据仓库到用户画像，从数据建设到数据应用

  olapdatawarehousesqlbigdataprestouserprofile
  614
• automate-dv

  Datavault-UK / automate-dv

  A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)

  datavault20dbtdatawarehousingdatavaultetlsnowflakedataengineeringdatawarehousedatalakeeltdata-vaultsqlmetadata
  564

• shunfei / indexr

  An open-source columnar data format designed for fast & realtime analytic with big data.

  indexrcolumnar-storageolapdatawarehouserealtime
  Language:Java 452
• yuniql

  rdagumampan / yuniql

  Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!

  sqlyuniqldatabase-migrationssqlserverpostgresqlmariadbmysqlredshiftsnowflakeamazon-rdsazure-sql-databasedatawarehousedata-engineeringdotnet-coredotnet-tooloracle
  Language:C# 424

• DataWithBaraa / sql-data-warehouse-project

  A comprehensive guide to building a modern data warehouse with SQL Server, including ETL processes, data modeling, and analytics.

  data-analysisdata-analyticsdata-cleaningdata-engineeringdata-lakehousedata-sciencedata-warehousedata-warehousingdatalakedatasciencedatawarehousedatawarehousingetletl-jobetl-pipelinemedallion-architecturesqlsql-querysql-serversqlserver
  Language:TSQL 405
• arc

  Basekick-Labs / arc

  High-performance time-series database. 2.4M metrics/sec + 950K logs/sec + 940K traces/sec + 940K events/sec. One endpoint, one protocol. DuckDB + Parquet + Arrow. AGPL-3.0

  databasedatawarehouseperformancetime-serieslogseventstraces
  Language:Python 361

• dataplane-app / dataplane

  Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.

  golangairflowkubernetesdataschedulerworkflowsworkflow-automationdata-pipelinesdata-sciencedata-engineeringworkflowdata-analysisdata-integrationdataplanedatawarehouseetlpipelinesrobotics-process-automationrpafinance
  Language:JavaScript 356

• DragonKingpin / Hydra

  Hydra九头龙，面向PB级别知识库取数、情报系统、数据平台、大规模控制调度系统。面向大规模数据采集、分析、智能取数。——以实现大规模分布式爬虫搜索引擎为例。

  bigdatacloudcrawlerdatawarehousedistributed-systemsinfrastructuremicroservicesearch-enginearchitectureoperating-systemknowledge-graphquant
  Language:Java 329

• memiiso / opendbt

  Make dbt great again! Extend dbt with plugins, local docs and custom adapters — fast, safe, and developer-friendly

  analyticsdatawarehousedbtdbt-coreetlforkhacktoberfesthacktoberfest2025open-sourcesql
  Language:Python 243
• Data-Engineering-Roadmap

  ErdemOzgen / Data-Engineering-Roadmap

  Roadmap for Data Engineering

  awesomeci-cdclouddata-sciencedataengineeringdevelopmentdevopsmachine-learningmlopsroadmapdatabasedatapipelinedatapreprocessingdatawarehousedeep-learninginterviewawesome-listawesome-resourcesguidelines
  Language:Java 237

• cuebook / CueObserve

  Timeseries Anomaly detection and Root Cause Analysis on data in SQL data warehouses and databases

  sqlanomalyanomaly-detectionbigqueryredshiftsnowflakedatawarehousetimeseries-analysistimeseries-forecastingprophet-facebookroot-cause-analysis
  Language:Python 232

• jitsucom / bulker

  Service for bulk-loading data to databases with automatic schema management (Redshift, Snowflake, BigQuery, ClickHouse, Postgres, MySQL)

  data-engineeringdatawarehouseetletl-pipelineingestionpipeline
  Language:Go 204

• josephmachado / simple_dbt_project

  Code for dbt tutorial

  tutorialdbtpostgresdatawarehouseanalytics
  161

• zhongyu09 / openchatbi

  OpenChatBI is an intelligent chat-based BI tool powered by large language models, designed to help users query, analyze, and visualize data through natural language conversations. It uses LangGraph and LangChain to build chat agent and workflows that support natural language to SQL conversion and data analysis.

  agentaianalyticsbidatabasedatawarehousegptlangchainlanggraphllmnlptext2sql
  Language:Python 135

• neuralinkcorp / datarepo

  data-warehousedatalakedatawarehousedelta-lake
  Language:Python 133

• bennyaustin / fabric-accelerator

  Accelerator to build a Microsoft Fabric modern data platform using pre-built reusable Fabric items and an orchestration ELT Framework

  acceleratordata-factorydatawarehouseelt-frameworkiac-biceplakehousemicrosoft-fabricpipelinespark-notebookeventstreamkustoreal-time-intelligence
  Language:Jupyter Notebook 106

• locnd-172 / IBM-Data-Engineer-Specialization-Coursera-Personal-Note-Public

  All of my individual learning materials, documents, and notes from the process of getting the Coursera IBM Data Engineer Professional Certificate specialization are stored in this repository.

  courseradata-engineeringdatawarehouseetlibm
  Language:Jupyter Notebook 105

• samber / awesome-olap

  A curated list of awesome Online Analytical Processing databases, frameworks, ressources and other awesomeness.

  analyticsawesomeawesome-listdatabasedatalakelakehousemetastoreolapprocessingsqlbatch-processingbig-datadatawarehousestream-processingstreamingtimeseries
  96

• MohamedHmini / tweetsOLAPing

  implementing an end-to-end tweets ETL/Analysis pipeline.

  datawarehousingdatawarehouseetl-pipelinetweetstweets-classificationtweets-scrapertwitter-apigoogle-api-clientapi-clientweb-crawlinganalysiscube-analysismulti-dimensional-analysisssisssas-multidimensionalmultithreadingpowerbi-report
  Language:Python 58

• josephmachado / online_store

  End to end data engineering project

  python3data-engineeringdata-pipelinedagsterpostgresqldatawarehousedata-processingdata-platformdata-analysis
  Language:Python 57

• Matts966 / alphasql

  AlphaSQL provides Integrated Type and Schema Check and Parallelization for SQL file set mainly for BigQuery

  bigqueryzetasqltypecheckerparallelizationgraphvizdot-languagedatawarehousedatawarehouseautomationdockerworkflow-automationworkflow-toolpipeline-frameworkdependency-graphdependency-analysiseltcycle-detection
  Language:C++ 56
• pyomop

  dermatologist / pyomop

  Python package for managing OHDSI clinical data models. Includes support for LLM based plain text queries, MCP server and FHIR import.

  ohdsidatawarehousehealth-informaticshealth-data-analysisclinical-trialscdmpythonhacktoberfestllmtext-to-sql
  Language:Python 52

• data-solution-automation-engine / data-solution-framework

  A library for data warehouse and data integration pattern and architecture documentation.

  architecturesolutionpatternsdesigndatawarehouseetldata-warehousesetl-processesetl-control
  51

• hifxit / dataligo

  A library to accelerate ML and ETL pipeline by connecting all data sources

  databasedatalakedatawarehouseetl-pipelineml-pipelinenosqlpython
  Language:Python 47

• data-solution-automation-engine / virtual-data-warehouse

  The Virtual Data Warehouse is a code generation and template management tool. It is part of the data solution automation ecosystem - the 'engine' for data solution automation.

  codegenerationdatawarehouseautomationetldatawarehousedatadatavaultdatavault20virtualvirtual-data-warehouseetl-automation
  Language:Handlebars 45

• glynnbird / couchwarehouse

  Data warehouse for CouchDB

  couchdbdatawarehousesqlsqlitenodejsclimysqlpostgresqlelasticsearch
  Language:JavaScript 45

• umer7 / Data-Warehouse-Concepts-Design-and-Data-Integration

  Repo for Data Warehouse Concepts, Design, and Data Integration by University of Colorado System (coursera)(Notes,Assignments, quiz and research papers)

  datawarehousepentahodata-integrationdata-warehouseoracle
  44

• data-solution-automation-engine / data-warehouse-automation-metadata-schema

  Generic interface exchange format for Data Warehouse Automation and ETL generation.

  datawarehouseautomationdatawarehouseetl-automationmetadata-managementetlgeneration
  Language:C# 42

• silverton-io / snowflakecli

  A DuckDB-powered command line interface for Snowflake security, governance, operations, and cost optimization.

  datadata-engineeringdata-sciencedatawarehousesnowflakesnowflakedb
  Language:Python 41

• SharpData / SharpETL

  Write ETL using your favorite SQL dialects

  etletl-frameworkhivescalasparkspark-sqlsqldatawarehousedata-warehousebigdataflinkflink-sqlpaimon
  Language:Scala 36

• KennethanCeyer / awesome-data-pipeline

  Awesome list for datapipeline

  datadatapipelinebigdataopensourcedatalakedatawarehousehadoopsparkclouddataenghivequeryarchitectureawesomeawesome-listbig-datadata-engineering
  35

• TauWu / backend_learning_notes

  后端学习笔记，本项目存放了一些我阅读有关的技术类的书籍和部分源码阅读的笔记整理。 涉及范围包括后端开发中的计算机学科基础知识、高级语言的基础知识、源码阅读笔记、数据库知识、数据挖掘知识等，同时也会涉及到一些具体生产场景中会遇到的一些实际问题。 :-D

  data-structuresdatabasedatawarehousegolangjavaleetcodenetworkoperating-systempython3
  Language:Python 32

• Balajirvp / DE-Zoomcamp

  Code/Notes for the Data Engineering Zoomcamp by DataTalksClub

  batchprocessingbigquerydataengineeringdbtdockergcpprefectstreamingpysparkpythonrddsparksqlterraformdatalakedatamodelingdatawarehouse
  Language:Jupyter Notebook 31