Thiru-Perumal

streaming_data_processing

Create a streaming data, transfer it to Kafka, modify it with PySpark, take it to ElasticSearch and MinIO

ease-with-apache-spark

Series follows learning from Apache Spark (PySpark) with quick tips and workaround for daily problems in hand

Data-Pipelines-with-Airflow

This project helps me to understand the core concepts of Apache Airflow. I have created custom operators to perform tasks such as staging the data, filling the data warehouse, and running checks on the data quality as the final step. Automate the ETL pipeline and creation of data warehouse using Apache Airflow. Skills include: Using Airflow to automate ETL pipelines using Airflow, Python, Amazon Redshift. Writing custom operators to perform tasks such as staging data, filling the data warehouse, and validation through data quality checks. Transforming data from various sources into a star schema optimized for the analytics team’s use cases. Technologies used: Apache Airflow, S3, Amazon Redshift, Python.

machinelearning

griffin

Mirror of Apache griffin

sfguide-data-engineering-with-snowpark-python

dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.

ev-pattern-analysis

pandas_dq

Find data quality issues and clean your data in a single line of code with a Scikit-Learn compatible Transformer.

Practical-Python-Design-Paterns

Cloudera_Material

Cloudera_Material: Study Material to help people preparing for Cloudera CCA Spark and Hadoop Developer Exam (CCA175). Feel free to collaborate.

Stock-Prediction-Models

Gathers machine learning and deep learning models for Stock forecasting including trading bots and simulations

data-engineering-interview-questions

More than 2000+ Data engineer interview questions.

codewars-handbook

A code warrior's handbook 🐱‍💻

python-poetry-docker-example

Example of integrating Poetry with Docker leveraging multi-stage builds.

Hackerrank_Python_Domain_Solutions

Solutions of challenges of Hackerrank Python domain

Hackerrank-Problem-Solving-Python-Solutions

Hackerrank Problem solving solutions in Python

HackerRank-Solutions

HackerRank concepts & solutions

Hackerrank

Solutions to the practice exercises, coding challenges, and other problems on Hackerrank! www.Hackerrank.com

HackerRank

HackerRank solutions in Java/JS/Python/C++/C#

HackerrankPractice

170+ solutions to Hackerrank.com practice problems using Python 3, С++ and Oracle SQL

Hackerrank_Python_Solutions

HackerRank Python solutions and challenges.

codewars-python

spark

Apache Spark - A unified analytics engine for large-scale data processing

cheat-sheet

collection of cheat sheets

scenic

Scenic: A Jax Library for Computer Vision Research and Beyond

PythonDataIES

The course site for the Data Processing in Python from IES

dask

Parallel computing with task scheduling

python-machine-learning-book

The "Python Machine Learning (1st edition)" book code repository and info resource

best-of-ml-python

🏆 A ranked list of awesome machine learning Python libraries. Updated weekly.