magrathj

Pyspark-Docker-Image-With-Azure-Gen2-Connection

Building a docker image with Pyspark and Azure Gen2 storage connector so enable local testing your lake

NYC-Taxi-Analysis

Big data project to analyse 1.1 billion taxi trips in NYC and create a forecasting model

delta-migrations

Pyspark package to handle schema migrations in larger scale projects where incremental changes to tables are required.

Disaster-Response-NLP-Pipeline

Building a ML pipeline using the figure eight Corporate messaging dataset

arduino_devops

Arduino github actions workflow to validate changes

arduino_iot_device

arduino uno with GPS and Bluetooth

event-driven-hackathon

using event driving architecture for geo-location based promotios

automated-gcp-databricks-deployments

Automated deployments with Databricks GCP

Azure-Container-MLFlow-Model-Registry

Project looks to create a stand-alone MLflow model registry which sits on its own Azure Container Registry, using an image, connected to a blob storage (artifact store) and internal sqlite db (registry store).

Azure-Devops-ML-Flask-WebApp

Machine learning web app with CI/CD component using agile framework to build out the project.

Azure-Devops-Terraform-CI-CD-App-Service

Building out an Azure CI/CD pipeline to create disposable test environments and run automated tests on a published Web App

Azure-FlaskWebApp

Deploy An Article CMS To Azure

azure-web-form

Data-Integration-Pipelines-for-NYC-Payroll-Data-Analytics

The City of New York would like to develop a Data Analytics platform

Data-Modeling-with-Cassandra

Create a build and load process for a single VM with Cassandra

Data-Modeling-with-Postgres

Create a build and load process for a single VM with Postgres

Data-Science-Challenge

This challenge looks at using linux, docker, postgres and python to analyse interesting trends in transcational banking data.

databricks-dbt-gcp

Using dbt on GCP Databricks

DataOps

Example code for doing DataOps

dbt-snowflake

dbt snowflake set up demo

delta-live-tables-poc

PoC to test out Delta Live Tables with CI/CD pipeline deployments

delta-migrations-template

Deploying-Databricks-Notebooks-CI-CD

Using docker to deploy packages and notebooks using docker to an Azure Databricks workspace

fire

Framework to provide schemas and constraints for Delta Live Table code

Github-Network-Analysis-Web-App

Python app that uses oauth to query github api and returns a network graph of the users repos

migrate

Scripts to help customers with one-off migrations between Databricks workspaces.

snowflake-warehouse

Snowflake warehouse which uses weather and restaurants (YELP) datasets

snowflake_ci_cd

strava_app

Web app to display strava user data

Synapse-Warehouse

Example of Synapse data warehouse which pulls data from Azure SQL DB