rohitshubham / big_data_1

Assignment Assignment_01 801636

Structure of Assignment

The root directory of the assignment consists of 4 folders.

1. Code folder: Contains the source code of the assignment. The scrip folder consists of all the shell scripts for building and running the project.

2. Data : contains the file data.csv used for testing the setup and data ingestion. This is Airbnb's data in csv format.

3. Logs: Contains the log from the various components. Suggested to use Grep to search relevant data in these logs.

4. Report: It has the report including the design of the assignment. Basically it has 2 files: Report.md that has the design and other similar details about this assignment. And Assignment-01-Deployment.md that has the steps to deploy the whole project.

---

Miscellaneous details:

• This whole project was developed for a containerized environment.I have used docker and docker-compose on Ubuntu.

• The code was written in Python 3.6.

• The version control system used was git and the code is repo is on version.aalto.fi. It is hosted on https://version.aalto.fi/gitlab/rajr1/assignment_1_801636. Please contact the author to get access.