kk17 / extend_spark_application_example

Examples for Spark application Dependency Management

Examples for Spark application Dependency Management

see related blog post here

Run examples

Need Docker and Docker compose installed.

1. Package

sbt package

2. Run Spark Standalone Cluster

copy .env.example to .env

docker-compose up -d

3. Run test script

bash scripts/xxx.sh

Reference

java - Add jars to a Spark Job - spark-submit - Stack Overflow