lakehouse

There are 8 repositories under lakehouse topic.

• presto

  prestodb / presto

  The official home of the Presto distributed SQL query engine for big data

  javaprestohivehadoopbig-datasqldatalakehousequery
  Language:Java 15622

• apache / doris

  Apache Doris is an easy-to-use, high performance and unified analytics database.

  olapdatabasehadoophivehudiicebergreal-timesqlbigquerydbtdelta-lakeeltetllakehousequery-engineredshiftsnowflakespark
  Language:Java 11479

• StarRocks / starrocks

  StarRocks, a Linux Foundation project, is a next-generation sub-second MPP OLAP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics, and ad-hoc queries. InfoWorld’s 2023 BOSSIE Award for best open source software.

  databaseolapsqlanalyticsbig-datarealtime-databasevectorizeddistributed-databasereal-time-analyticsmppjoinstar-schemareal-time-updatesdelta-lakehudiiceberglakehousedatalakelakehouse-platformcloudnative
  Language:Java 7973

• paradedb / paradedb

  Postgres for Search and Analytics

  aggregationsanalyticsbig-databm25databasedatalakeelasticsearchfull-text-searchhtaphybrid-searchlakehouse-platformmppobject-storageolappostgresqlreal-time-analyticsself-hostedsimilarity-searchsparse-vectorsql
  Language:Rust 4202

• lakesoul-io / LakeSoul

  LakeSoul is an end-to-end, realtime and cloud native Lakehouse framework with fast data ingestion, concurrent update and incremental data analytics on cloud storages for both BI and AI applications.

  lakesouldatalakelakehousesparkflinkstreamingbig-datapostgresqlrustsqlhuggingfacepythonpytorcharrowdatafusionvectorizedvelox
  Language:Java 2311

• ytsaurus / ytsaurus

  YTsaurus is a scalable and fault-tolerant open-source big data platform.

  big-dataclickhousedistributed-databaselakehouseolap-databasesparksqlytsaurus
  Language:C++ 1772

• ByConity / ByConity

  ByConity is an open source cloud data warehouse

  bytedanceclickhouseclickhouse-databasecloudkubernetslakehouseolaps3snowflakesqltiktok
  Language:C++ 1609

• apache / amoro

  Apache Amoro (incubating) is a Lakehouse management system built on open data lake formats.

  bigdatadatalakelakehouse
  Language:Java 712

• datastrato / gravitino

  World's most powerful data catalog service with providing a high-performance, geo-distributed and federated metadata lake.

  datalakelakehousemetadatafederated-querystratospheremetalakeskycomputingdata-catalogai-catalogmodel-catalog
  Language:Java 343

• cuebook / cuelake

  Use SQL to build ELT pipelines on a data lakehouse.

  apache-icebergdeltalakehousedatalakedata-lakeeltetldata-engineeringdata-integrationdata-ingestionapache-sparkspark-sqlupsertincremental-updatesdata-transferpipelinesdata-pipelinezeppelin-notebooksql
  Language:JavaScript 283

• qinsql / QinSQL

  AI 时代的智能数据库

  olapoltplakehouse
  Language:Java 226
• dataall

  data-dot-all / dataall

  A modern data marketplace that makes collaboration among diverse users (like business, analysts and engineers) easier, increasing efficiency and agility in data projects on AWS.

  aws-s3datalakehouseredshiftlakeformationaws-glueaws-lake-formationdata-scienceetl-frameworkaws
  Language:Python 212

• adidas / lakehouse-engine

  The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.

  big-dataconfiguration-drivendata-engineeringdata-qualitydatabricksdelta-lakeframeworkgreat-expectationslakehousespark
  Language:Python 185

• databricks / terraform-databricks-examples

  Examples of using Terraform to deploy Databricks resources

  awsazuredatabricksgcplakehouseterraformterraform-moduledatabricks-module
  Language:HCL 183

• icelake-io / icelake

  Pure Rust Iceberg Implementation

  iceberglakehouserust
  Language:Rust 161

• google / space

  Unified storage framework for the entire machine learning lifecycle

  datasetlakehousemachine-learningmlopsolapraytensorflowtensorflow-datasetapache-arrowapache-parquetdata-warehousedataopsdmlmultimodalmultimodal-data
  Language:Python 137

• pracdata / awesome-open-source-data-engineering

  A curated list of open source tools used in analytical stacks and data engineering ecosystem

  awesomeawesome-listdata-analyticsdata-engineeringdata-platformdatabaseself-hostedmlopsdata-storagedatadata-integrationdata-lakehousedatalakelakehouseworkflow-engineanalyticsdata-warehouseobservabilitydata-pipelineetl
  61

• lhbench / lhbench

  Lakehouse storage system benchmark

  apache-hudiapache-iceberglakehousebenchmarkcidrdatabasedatabricksdelta-lake
  Language:Scala 57

• dominikhei / Local-Data-LakeHouse

  Sample Data Lakehouse deployed in Docker containers using Apache Iceberg, Minio, Trino and a Hive Metastore. Can be used for local testing.

  apache-icebergdata-lakedata-lakehousehive-metastorelakehouseminiotrino
  Language:Dockerfile 45

• databrickslabs / delta-oms

  DeltaOMS is a solution that help build a centralized repository of Delta Transaction logs and associated operational metrics/statistics for your Delta Lakehouse. Unity Catalog supported in the v0.7.0-rc1 release.Documentation here - https://databrickslabs.github.io/delta-oms/v0.7.0-rc1/

  centralizeddatabricksdeltadelta-lakelakehousemetricsmonitoring
  Language:Scala 36

• ysfesr / Building-Data-LakeHouse

  Creation of a data lakehouse and an ELT pipeline to enable the efficient analysis and use of data

  sparkdelta-lakedockerhivelakehouseminioprestos3-storage
  Language:Python 33

• leehuwuj / olh

  Open source stack lakehouse

  bigdatadataplatformdeltalakekuberneteslakehousespark
  Language:Python 23

• samber / awesome-olap

  A curated list of awesome Online Analytical Processing databases, frameworks, ressources and other awesomeness.

  analyticsawesomeawesome-listdatabasedatalakelakehousemetastoreolapprocessingsqlbatch-processingbig-datadatawarehousestream-processingstreamingtimeseries
  21

• vvalcristina / Workshop-Data-Lakehouse

  Repositório dedicado a Workshop de Data Lakehouse com Delta Lake

  sparkdelta-lakelakehouse
  Language:Jupyter Notebook 18

• databricks-industry-solutions / omop-cdm

  Unlocking the Power of Health Data With a Modern Data Lakehouse

  databricks-industry-solutionshlslakehouseomop-cdmrwe
  Language:Python 17

• microsoft / Fabric-RTA-FlightStream

  Microsoft Fabric Real-time Analytics flight streaming

  etlkqllakehousepowerbipyspark-notebookstreaming
  Language:Jupyter Notebook 16

• prestodb / prestorials

  Tutorials and examples of how to deploy Presto and connect it to different data sources

  awsawsgluedatadatalakedockerexamplegluelakehousemongodbprestopresto-connectorprestodbprestosqlsqltutorialwalkthrough
  15

• tspannhw / FLiPStackWeekly

  FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...

  apacheflinkapacheicebergapachekafkaapachenifiapachesparkclouderalakehousestreamingtimspann
  14

• apache / doris-streamloader

  Stream Loader for Apache Doris

  bigquerydatabasedbtdelta-lakeeltetlhadoophivehudiiceberglakehouseolapquery-enginereal-timeredshiftsnowflakesparksql
  Language:Go 13

• victorskl / genomic-bigdata-spark

  Genomic BigData Warehousing with Apache Spark and LakeHouse Architecture

  bigdatabioinformaticsdatalakedelta-lakegenomics-datalakehouseparquetsparkspark-sqldatawarehousingcloud-computingcloudnative
  Language:Jupyter Notebook 11

• databrickslabs / waterbear

  Automated provisioning of an industry Lakehouse with enterprise data model

  data-modellakehousesparkdelta-lakepythondatabrickssql
  Language:Python 9

• ekote / Build-Your-First-End-to-End-Lakehouse-Solution

  Build Your First End-to-End Lakehouse Solution (aka.ms/fabconlake)

  apache-sparkdata-factorydata-pipelinedata-sciencedelta-lakelakehousemachine-learningmicrosoft-azuremicrosoft-fabricparquetpowerbiwarehousedata-engineeringdataflowstutorialworkshop
  Language:Jupyter Notebook 9

• AnthonyByansi / MicrosoftFabric-Exploratorium

  A comprehensive educational resource hub dedicated to mastering Microsoft Fabric, offering in-depth tutorials, real-world use cases, and hands-on guides for seamless end-to-end analytics

  analyticsbig-datacollaborationdata-analyticsdata-engineeringdata-integrationdata-sciencedata-transformationdata-warehousingdelta-lakelakehousemicrosoft-fabricone-lakepowerbireal-time-analyticsserverless-computingspark
  Language:Shell 8

• databricks-industry-solutions / dns-analytics

  Leverage the Databricks Solution Accelerator for DNS analytics to accelerate time to detection and response across petabytes of data. Tap into DNS traffic logs, enrich streaming threat intelligence, and apply advanced analytics to detect DNS abnormalities and prevent malicious attacks.

  cybersecuritydatabricks-industry-solutionsdnslakehousestreaming
  Language:Python 8

• ComputeAI / computeAI-integrations

  Supercharge Your Compute for Analytics & AI

  analyticsdata-warehousedatabaseexternal-tablehive-tableiceberglakehouseolapparquetsql
  Language:Jupyter Notebook 7

• databricks-industry-solutions / factory-optimization

  Overall Equipment Effectiveness: Performant and Scalable End-to-End Equipment Monitoring

  databricks-industry-solutionsiotlakehousemfg
  Language:Python 7