xbzheng / de_tools_dev_setup

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Test Suites for Installation of Distributed Computing Tools

Author: Ivan Zheng Last update: 6/16/2019

Background

Insight Data Engineering offers a 7-week, intensive professional training fellowship program to help software engineers and academic programmers to make a career transition to data engineering.

At the program, the Fellows build proof-of-concept data platforms using open source, distributed technologies. Due to the rapid development of the technologies, the dev setup of each tool often needs an update every 2 - 3 months. This repo provides sample codes to test the functionality of tools as they get updated and covers the technologies below in the Data Engineering Ecosystems.

  • Apache Spark
  • Apachel Kafka
  • PostgreSQL
  • Flask

About


Languages

Language:Jupyter Notebook 94.9%Language:Python 5.1%