shivananda199 / aws-etl-pipeline-on-youtube-data

To build an end-to-end data pipeline on YouTube data using AWS services

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

aws-etl-data-pipeline

Learn how to build ETL data pipeline in Python on YouTube data using various AWS services.

Prerequisites

Few things you need to have before starting the project:

  • Understanding of AWS services: EC2, IAM, Glue, Lambda, Athena, S3
  • Good knowledge of SQL, shell scripting, and Python

Project Motivation

The main motive behind the project is to understand building an end-to-end data pipeline (extract, transform, load, and visualize) using AWS services.

Architecture Diagram

Data_Architecture_Diagram

About

To build an end-to-end data pipeline on YouTube data using AWS services


Languages

Language:Python 74.7%Language:Shell 25.3%