Otacilio Filho's repositories
spark-dev-env-docker
Spark development environment for kubernetes, spark-submit and jupyter notebook
azure-cloud-handler
Python library to interact with some resources on Azure as AKS (Azure kubernetes service) and Data Lake Storage Gen2
azure-spark-on-kubernetes
Spark on Kubernetes with Azure resources: Azure Kubernetes Service (AKS), Azure Data Lake Storage Gen2 and Azure Synapse
azure-functions-ingestion
Example of how to use Azure Functions to Ingest an API data to datalake
data-academy
project repo to work in Azure enviroment
igti-cde-mod3-desafio
Challenge solved using Google Cloud Platform Dataproc and Storage services
data-eng-open-source-tools
Explore and test Open-sources tools for Data Engineering in standalone mode or integrated with a ecosystem
ingestion-on-postgres
Ingestion of NY Taxy data on Postgres Database with Python or Spark
lineage-keeper
A lightweight lineage tool based on Spark and Delta Lake
data-engineering-roadmap
roadmap de engenharia de dados da jornada 2024
DeltaLakeReader
Read Delta tables without any Spark
desafio-stone-dataengineer
Repositório com a solução proposta para o desafio da Stone de Data Engineer
dev_api_with_flask
Developing an REST API with Flask MicroFramework
docker-bigdata
Big Data Ecosystem Docker
docker-nosql
Differents NoSql Databases with docker: mongoDB and Redis
DockerSwarm-MinIO
Deploy MinIO storage server in Docker Swarm
igti-cde-mod3-trab-pratico
Trabalho prático modulo 3 Cloud Data Engineer IGTI
igti-cde-practice-one-aws
Practice one from Cloud Data Engineer by IGTI
kafka-samples
Kafka Lab done with docker
otacilio-psf.github.io
Portifolio
pyspark-codespace
Template in how to use PySpark + DeltaLake + Azure Storage + test cases in Codespace
spark-image
Spark image built-in with connectors to Common data sources and Delta lake
workshop-dw-pagando-pouco
workshop 03 - como montar um dw pagando pouco