scotthaleen / spark-saturday

Workshop for Spark and Databricks

Spark Saturday July 27.

Jumpstart on Apache Spark 2.2 with Databricks

Agenda

Morning

• Get to know Databricks
  • Notebook 0: WordCount
• Overview of Spark Fundamentals & Architecture
• What’s New in Spark 2.x
• Break
• Unified APIs: SparkSessions, SQL, DataFrames, Datasets…
• Workshop Notebook 1: SparkSession
• Lunch

Afternoon

• Introduction to DataFrames, Datasets and Spark SQL
• Workshop Notebook 3: IoT and Datasets
• Break
• Introduction to Structured Streaming Concepts
• Workshop Notebook 4: IoT and Structured Streaming

Instructions to Register for Free Databricks Community Edition

• Go to http://databricks.com/try-databricks

• Start Today for Community Edition.

• Make sure you use an email address from which you can access e-mails.

• Got to gitbub: https://github.com/dmatrix/spark-saturday

• Download DBC file: MeetupWorkshops.dbc

• Go to your Databricks-->Workspace->Users->[e-mail]->Import

  • Click File option
  • Click on "Drop file here to upload or click to select."
  • Import MeetupWorkshops.dbc

You should have folder by that name with all the notesbooks

Notebooks URLS for the Labs:

• Word Count: http://dbricks.co/ss_wkshp0
• Spark Session: http://dbricks.co/ss_wkshp1
• SQL & Datasets (optional): http://dbricks.co/sqlds_wkshp2
• DataFrames & SQL (optional): http://dbricks.co/sqldf_wkshp2
• Mount Points (python): http://dbricks.co/data_mounts
• Datasets & IoT Devices: http://dbricks.co/iotds_wkshp3
• Streaming & IoT Devices: http://dbricks.co/iotss_wkshp4

Resources and APIs

• http://docs.databricks.com
• http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.SparkSession
• https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.Dataset