datafactory
There are 2 repositories under datafactory topic.
Azure-Samples / modern-data-warehouse-dataops
DataOps for Microsoft Data Platform technologies. https://aka.ms/dataops-repo
Language:Shell 660rlevchenko / terraform-azure-data
Terraform script to deploy almost all Azure Data Services
Language:HCL 37DataTech-Solutions / Threat-Detection-and-Visualization
Threat Detection and Visualization
Language:TSQL 32mehroosali / databricks-F1-Project
A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.
Language:Jupyter Notebook 12guderkar / ACIProxy
Azure Container Instances Proxy implemented in Azure Function App (Consumption Plan)
Language:Python 10NandanHegde15 / Azure-DataFactory-Generic-Pipelines
Generic Pipelines / Templates for Data Factory / Synapse Pipelines w.r.t Different MSFT Offering Integrations / Use Cases
- demo-mdwh
camposvinicius / azure-etl
This pipeline is an ETL application, which aims to collect data for a few minutes from Binance's open API for cryptocurrencies: BTCUSDT, ADAUSDT, ETHUSDT, BNBUSDT and LTCUSDT.
Language:Smarty 4shubhammirajkar / tokyo_olympics_de_project
Explore the Tokyo Olympics data journey! We ingested a GitHub CSV into Azure via Data Factory, stored it in Data Lake Storage Gen2, performed transformations in Databricks, conducted advanced analytics in Azure Synapse, and visualized insights in Synapse or Power BI.
Language:Jupyter Notebook 4SixGod191001 / azure-adf-framework
The ADF Universal Framework is an open-source project designed to provide a comprehensive and flexible solution for building scalable and efficient data integration workflows using Azure Data Factory (ADF).
Language:TSQL 4federicopfund / data-engineer
Proceso ETL
Language:Jupyter Notebook 3tknishh / olympic-data-analysis-azure
End-to-End data engineering project with Azure Databricks as cloud service and Tokyo olympic data
Language:Jupyter Notebook 3snhaider9977 / Datafactory-CI-CD
This repository serves as a sample demonstration of implementing CI/CD practices on Azure Data Factory using Azure DevOps. It provides an example approach to configure a pipeline for continuous integration and continuous deployment (CI/CD) workflows on Azure Data Factory
- Language:PowerShell 1
- Language:Java 1
ayanhussain81 / Olympics-Data-ETL
Tokyo Olympic Azure Data Engineering Project
Language:Jupyter Notebook 1bennyaustin / ADFCustomActivity-AASPartition
Repository for Azure Data Factory (ADF) Custom Activity to dynamically create and process Azure Analysis Service (AAS) Tabular Model Partitions
Language:C# 1Logic App that calls Azure Resource Management REST API to get information about resources in a subscription
Language:PowerShell 1BrioGit / DataFactoryAutomation
This application shows how to use the Azure .NET library to create and execute Azure Data Factory from C# code.
Language:C# 1liliankotvan / Azure-ADF-RefreshPBI
Azure Data Factory template to refresh Power BI Dataset
MekWiset / Medallion_DataLakehouse
Building a Data Lakehouse using the Medallion architecture.
Language:Jupyter Notebook 1MuhammadHasaanWahid / Data-Filtering-Pipeline-ETL
This Project Extracts supply chain data from csv file having 180k records and more than 40 columns from the Azure Datalake Gen2 storage account and do some dataanalysis with Python(Pandas) to find the top 3 countries and filtered the data for top 3 countries and finally transferred it to 3 files in datalake again by creating ETL pipeline in ADF.
Language:Jupyter Notebook 1thanaphongK37 / Data-Science-and-Data-Analyst-Project
Portfolio Data Analysis and Data Science projects and Data Engineer built using Azure Service, SQL and Python.
Language:Jupyter Notebook 1tuancamtbtx / Azure-LandingZone-Demo
Azure Data Landing Zone - IAC , Config, Code
Language:HCL 1venkatakamaiah46 / Python
Interesting_programs_written_in_Python_language
AsadAhmed29 / End-to-End_ETL_pipeline
In this project, I built a comprehensive data pipeline using Azure services to address a business need for insights on customer demographics and sales trends. Data from an on-premises SQL database is ingested, transformed, and visualized in a Power BI dashboard.
Language:Jupyter Notebook 0- Language:Jupyter Notebook 0
Maaztajmohammed / NYC-Taxi-Data-Engineering-Project
This project builds a cloud-based pipeline to extract NYC taxi data from an API and store it in Azure Data Lake Storage (ADLS). Databricks and PySpark are used to transform the data through the medallion architecture (Bronze → Silver → Gold). Delta Lake ensures reliable storage, and Power BI provides visual insights for data-driven decision-making.
Language:Jupyter Notebook 0Punam918 / Adventurework_dataengineering
Azure data engineering project including data ingestion, storage, transformation, and visualization.
Language:Jupyter Notebook 0ryanninodizon / EHR-PoC
End-to-End Concept Project: Azure Data Factory Pipeline, Azure IoT Central, Azure Storage Account, Azure SQL, .NET, Angular
Language:TypeScript 0- Language:Jupyter Notebook
Saikesana31 / Adventure_Works_DE
Azure Data engineering project
Azure Data engineering project
Language:Pythonvskvj3 / End-to-End-Pipeline-for-Swiggy-Restaurant-Data
This project develops a data engineering pipeline to analyze restaurant data from various cities on the Swiggy platform. Using PySpark, Spark SQL, and Azure Data Factory, the data is processed and transformed to generate insights on ratings, cuisines, and trends, presented through dashboards and reports.
Language:Jupyter Notebook
