muralipoola / scala-demo

HelloWorld Spark Application

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

####HelloWorld Example

spark-submit --master local  --class learning.spark.HelloWorld ./target/scala-2.11/scala-demo_2.11-0.1.jar 

####File Formats Example The spark-avro module is external and not included in spark-submit or spark-shell by default.

As with any Spark applications, spark-submit is used to launch your application. spark-avro_2.11 and its dependencies can be directly added to spark-submit using --packages, such as,

spark-submit --master local  --packages org.apache.spark:spark-avro_2.11:2.4.0 --class learning.spark.BigDataFileFormats ./target/scala-2.11/scala-demo_2.11-0.1.jar

####WordCount Example using RDD's

spark-submit --master local  --class learning.spark.rdd.WordCount ./target/scala-2.11/scala-demo_2.11-0.1.jar ./ReadMe.md ./counts

About

HelloWorld Spark Application


Languages

Language:Scala 100.0%