samuelmaina / data-analysis-with-pyspark

Perfom data analysis using pyspark. Covers Spark functions(trigonometric functions, windows and lags), SQL views and queries and Parallelizing Spark operations.

Data-Analysis-with-pyspark

Perform data analysis using pyspark. Skills learnt incude:

1. Reading and loading data into data frames, use of RDD,
2. Use of SQL views in Spark.
3. Parallellizing data querying and processing.

• Specification of the problem can be found at Data Analysis with Spark (PySpark).docx