rogeriomm

aws-lab

Terraform AWS free tier, EC2/ECR/RDS/EFS/DynamoDB/Lambda/S3. Docker running on EC2, Traefik reverse proxy, Lets Encrypt, dynamic DNS, OpenVPN(TODO)

labtools-k8s

Complete data engineering pipeline running on Minikube Kubernetes, Argo CD, Spark, Trino, S3, Delta lake, Postgres+ Debezium CDC, MySQL,Airflow, Kafka Strimzi, Datahub, OpenMetadata,Zeppelin, Jupyter, JFrog Container Registry

localstack-k8s

Localstack

minikube-labtools-k8s

Minikube Kubernetes cluster(1+3 nodes) setup on macOS with Ingress,local DNS,NFS subdir external provisioner. Optimized to work on Intel Core i9-14900K with 128GB of RAM

challenge-delta-lake-deep-dive

The challenge to building your own Data Lakehouse using Delta Lake.

openshift-proxmox-terraform

Redhat Openshift(OKD Kubernetes) cluster install on Promox hypervisor using Terraform/Ansible

spark-k8s

Apache Spark + Kubernetes

stream-ingestion-redpanda-minio

In this article, you will learn how to set up a real-time data processing and analytics environment using Docker, MySQL, Redpanda, MinIO, and Apache Spark.

zeppelin-k8s

Apache Zeppelin running on Kubernetes

airflow-k8s

algorithms

kafka-k8s

Apache Kafka + Minikube

ansible-okd-proxmox

Ansible playbook and roles for easy install OKD on Proxmox using qcow2 images and templates.

azure-datafactory-databricks-lab

Azure Datafactory, Databricks lab, Terraform

build_api_devops_pipeline

data-engineer-handbook

This is a repo with links to everything you'd ever want to learn about data engineering

debezium-cdc-replication-delta

Replicating CDC Data to Delta Lake Using Apache Spark and Scala Engine with Debezium

demo-dbt-analytics-owshq

elixir-web-sample

Use Docker, Traefik reverse proxy + Let's encrypt running on ARM to create an Elixir/Phoenix development environment

hive-metastore-postgres-docker

hybrid-cpu

labtools-k8s-notebooks

labtools-k8s-terraform-talos

minikube

Run Kubernetes locally

public-server

Public Internet server based on Orange Pi 5 running Traefik Reverse Proxy + Let's Encript & Docker containers

public-server-monitoring

spark-airflow-zeppelin-twiter-s3-datalake

Apache Spark + Apache Airflow + Apache Zeppelin + S3 MINIO datalake twitter

ws-airflow-astro-python-sdk

Developing Modern ETL Pipelines on Apache Airflow with Astro Python SDK by Luan Moreno & Tatiana Al-Chueyr

ws-delta-lake-deep-dive

Delta Lake Deep Dive para Construção de um Data Lakehouse na Prática

zeppelin

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.