a49a / bigdata-sql-benchmark

Flink, Presto, Trino TPC-DS benchmark

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Big Data SQL Benchmark

This tool tests big data components by running some TPC-DS SQL.

Project Structure

hive-tpcds-setup

TPC-DS data generator. If you want to test Hudi or Iceberg, you can use official tools to convert this data to Hudi or Iceberg. Hudi tool: https://hudi.apache.org/docs/migration_guide Iceberg tool: https://iceberg.apache.org/docs/latest/spark-procedures/#migrate

jdbc-common

It tests TPC-DS via JDBC. You can run an Apache Kyuubi to expose JDBC service of Spark or Flink.

spark-hudi

It tests TPC-DS SQL via a Spark SQL jar job.

Acknowledgements

Inspired by https://github.com/ververica/flink-sql-benchmark

About

Flink, Presto, Trino TPC-DS benchmark

License:Apache License 2.0


Languages

Language:Java 92.6%Language:Shell 4.6%Language:Scala 1.7%Language:Makefile 1.1%