ven2day / building-lakehouse

Building Data Lakehouse by open source technology. Support end to end data pipeline, from source data on AWS S3 to Lakehouse, visualize and recommend app.

lakehouse on local

System Architecture

Lakehouse Architecture

Prepare infrastructure

docker compose up -d

Prepare data

• Upload data to MinIO, see README

Orchestrate pipeline

• Orchestrate pipeline, see README

Query data

• Query data, query on metabase, see README

Clean up

docker compose down

References

• Lakehouse
• Lakehouse Architecture
• Lakehouse: A New Generation of Open Platform for Data Analytics