iimpulse / spark-scala

Testing spark-scala with gradle

Spark-Scala-Demo

A barebones project with scala, apache spark built using gradle. Spark-shell provides spark and sc variables pre-initialised, here I did the same using a scala trait that you can extend.

Credit To: https://github.com/faizanahemad/spark-gradle-template.git`

Prerequisites

• Java
• Gradle
• Scala

Install

brew install apache-spark

Scala Plug In for Intellij

Clone the Repo

git clone ssh://[e-mail]:7999/tes/spark-scala.git

Build

./gradlew clean build

Run

./gradlew run

All Together

./gradlew clean run

Using this Repo

Import External Spark Library Jars from SPARK_HOME/libexec/jars/

Useful Links

• Spark Docs - Root Page
• Spark Programming Guide
• Spark Latest API docs
• Scala API Docs