Big-Data-Engineering
This repository contains the homeworks and other staff regards to the Big Data Engineering course (AY 21/22) at the University of Naples Federico II.
Homeworks
All the homeworks have been developed in team of 2.
- Homework1-MongoDB: design and development of a NoSQL database using MongoDB Compass for the storing of Yelp Dataset collections.
- Homework2-ApachePig: processing of Yelp Dataset Reviews collection using Apache Pig, with Pig Latin language.
- Homework3-ApacheSpark: distributed processing using Spark (PySpark) with support of Google Colab for the analysis of Yelp Dataset collections.
- HomeworkFinale-KPMG: data analysis of MIUR and ISTAT open dataset on university students enrolling using Python for the pre-processing of the data, MongoDB for the storage, and Apache Spark for the analysis.
KPMG Hackaton
- KPMG-UniversityTrends: a Python elaboration of MIUR and ISTAT open dataset, and an analysis of university trends with Pandas DataFrame, with development of some dashboards in Microsoft Power BI.