Hello! My name is Jennifer Ebe, I am a data engineer with over 5 years experience and this is my attempt at a senior data engineer code challenge
Using MySQL's public employee sample database, create a DAG to move data from the employee's table to BigQuery. There should also be two different dags, which would have different logic. The first dag should be a full load dag, which moves all the data from the employee's table to BigQuery, while the second dag would be an incremental dag, which would move data daily and only contain changes that have happened in the past day.
This repo contains the dag which answers the questions above
Connect with me Medium - https://medium.com/@ebejennifer LinkedIn - https://www.linkedin.com/in/jennifer-ebe/ Twitter/X - https://twitter.com/jennifer_ebe