jmitterh / ETL_Project

Extract, Transform, and Load

ETL Project

Extract, Transform, Load:

• Extract is the process of reading data from a dataset. In this stage, the data is collected, often from multiple and different types of sources.

• Transform is the process of converting the extracted data from its previous form into the form it needs to be in so that it can be placed into storage, such as a database. Transformation occurs by using rules or lookup tables or by combining the data with other data.

• Load is the process of writing the data into the target database.

Project Members:

• Jean-Paul Mitterhofer
• Baoxing Wang

Datasets you intend to use:

• Yahoo Finance API: https://finance.yahoo.com/quotes/API,Documentation/view/v1/
• Weather csv data: https://www.kaggle.com/berkeleyearth/climate-change-earth-surface-temperature-data#GlobalLandTemperaturesByCountry.csv
• Natural Disasters data: https://en.wikipedia.org/wiki/List_of_natural_disasters_in_the_United_States

What useful investigation could be done with the final database:

What effect does weather have with the United States economy based on the stock market index of large company performances? The final database can tell us the correlation between historical weather, natural disaster, and historical stock market behavior, with the common variable of time.

Whether final database will be relational or non-relational. Why

Relational Database. We have data that can be normalized to query the data. We also can relate all data with time.