Major Course Output #1 for the course Advanced Database Systems (STADVDB).
The course project on Query Processing in a Data Warehouse is a venue for students to integrate their knowledge and skills to achieve the following learning competencies:
- Build a data warehouse. Design the data warehouse following either the Star or the Snowflake schema; Include restructuring of tables and denormalization as needed.
- Setup an ETL script to load the data from source to the warehouse; Include Data Wrangling tasks, e.g. cleaning, splitting, merging, aggregating.
- Develop an OLAP application. Design your dashboard to display analytical reports generated using the data from the warehouse by applying OLAP operations, namely roll-up, drill-down, slice and dice.
- Apply query processing and optimization strategies to improve the performance of the application.
Students will use a public movies dataset from IMDB to (1) design a dimensional model for their data warehouse; (2) extract, transform and load the data from this dataset to their model using an ETL tool; and (3) formulate queries showing roll-up, drill-down, dice and slice operations to display the results in a visual dashboard. (4) Strategies in improving query processing speed through proper query formulation, the use of indexes, and database restructuring will be evaluated to determine how these affect the response time of the application.