Nazareno95

The goal of this project is to track the expenses of Uber Rides and Uber Eats through data Engineering processes using technologies such as Apache Airflow, AWS Redshift and Power BI.

Language:Jupyter NotebookApache-2.0111 7 3

udacity-data-eng-proj-1

Developed a data pipeline to automate data warehouse ETL by building custom airflow operators that handle the extraction, transformation, validation and loading of data from S3 -> Redshift -> S3

Language:Python89 80

airflow-provider-sample

A template repo for building and releasing Airflow provider packages.

Language:PythonApache-2.069 59 3

This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging the data, filling the data warehouse, and running checks on the data quality as the final step. Automate the ETL pipeline and creation of data warehouse using Apache Airflow. Skills include: Using Airflow to automate ETL pipelines using Airflow, Python, Amazon Redshift. Writing custom operators to perform tasks such as staging data, filling the data warehouse, and validation through data quality checks. Transforming data from various sources into a star schema optimized for the analytics team’s use cases. Technologies used: Apache Airflow, S3, Amazon Redshift, Python.

Language:Python69 10

nifi-workshop

26 90

udacity-data-eng-proj2

A production-grade data pipeline has been designed to automate the parsing of user search patterns to analyze user engagement. Extract data from S3, apply a series of transformations and load into S3 and Redshift.

Language:Jupyter Notebook24 20

Airflow-Tutorials-

Airflow Tutorials

Language:Python23 20

AWS-Serverless-Data-Engineering-Pipeline

This is a repository for the Duke University Cloud Computing course project on Serveless Data Engineering Pipeline. For this project, I recreated the below pipeline.

Language:Python18 20