murshidazher / ultimate-hands-on-hadoop

🐷 The ultimate hands on hadoop

ultimate-hands-on-hadoop

Hadoop with HDFS, YARN, MapReduce, Pig, Hive, Spark, Flink and more...

• This repository contains all the data, setup and execution scripts used during the coursework on Udemy - The Ultimate Hands on hadoop Tame your Big data
• Requires Hortonworks Sandbox 2.6.5, for docker script to run locally look into this repo.
• If you need terraform script to setup an sandbox environment in AWS, look into this repo.

Table of Contents

• ultimate-hands-on-hadoop
  • Table of Contents
  • Quick Navigation by Technology
  • Tech Stack
    • Core
    • Querying
    • Ingestion
    • NoSQL Databases
    • Streaming
    • Notebooks and Visualization
    • Other
  • License

Quick Navigation by Technology

• Introduction and ml-100k dataset
• MR using Python
• PigLatin and Pig
• Spark
• Hive
• HBase, MongoDB & Cassandra
• Drill, Phoenix, Presto
• Yarn, Tez, Mesos, Zookeeper, Oozie, Hue
• Real time Ingestion - kafka and flume
• Spark Streaming, Apache Storm & Apache Flink
• Impala, Apache Flume

Tech Stack

All the technologies used throughout the course

Core

• Introduction to the Hadoop Eco-system
• YARN
• HDFS
• MapReduce
• Pig
• Spark
• Hive
• Tez
• Mesos - Alternative cluster manager for YARN
• Zookeeper
• Oozie

Querying

• Apache Drill
• Apache Phoenix
• Presto

Ingestion

• Sqoop
• Kafka
• Flume

NoSQL Databases

• HBase
• Cassandra
• MongoDB

Streaming

• Spark Streaming
• Storm
• Flink

Notebooks and Visualization

• Apache Zeppelin
• Apache Superset

Other

• Impala
• Accumulo
• Redis
• Ignite
• Elasticsearch
• Kinesis
• Apache NiFi
• Falcon
• Apache Slider

License

MIT © Murshid Azher.