SparkPi
A simple Spark application for calculating the value of Pi using sbt and scala.
Pre-requisites
- Java
- Python
- Scala
- Spark
How to use?
- Install Scala plugin for Intellij IDEA.
- Install SBT plugin for Intellij IDEA.
- Import this project as a SBT project into your IntelliJ IDEA.
- Build and run the project.
How to build JAR?
> cd <project-directory>
> sbt compile
> sbt run
> sbt package
How to submit the Spark job to the local master?
spark-submit --class SparkPi --master local <path-to-the-jar>/<jar-file>.jar
Usually the path to the jar file is <project-directory>/target/scala-<scala-version>