datafactory
There are 2 repositories under datafactory topic.
Azure-Samples / modern-data-warehouse-dataops
DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.
Language:Shell 553rlevchenko / terraform-azure-data
Terraform script to deploy almost all Azure Data Services
Language:HCL 37DataTech-Solutions / Threat-Detection-and-Visualization
Threat Detection and Visualization
Language:TSQL 33guderkar / ACIProxy
Azure Container Instances Proxy implemented in Azure Function App (Consumption Plan)
Language:Python 10NandanHegde15 / Azure-DataFactory-Generic-Pipelines
Generic Pipelines / Templates for Data Factory / Synapse Pipelines w.r.t Different MSFT Offering Integrations / Use Cases
mehroosali / databricks-F1-Project
A data pipeline project build on databricks and azure to demostrate lifecycle of a cloud data project.
Language:Jupyter Notebook 5- demo-mdwh
camposvinicius / azure-etl
This pipeline is an ETL application, which aims to collect data for a few minutes from Binance's open API for cryptocurrencies: BTCUSDT, ADAUSDT, ETHUSDT, BNBUSDT and LTCUSDT.
Language:Smarty 3federicopfund / data-engineer
Proceso ETL
Language:Jupyter Notebook 3SixGod191001 / azure-adf-framework
The ADF Universal Framework is an open-source project designed to provide a comprehensive and flexible solution for building scalable and efficient data integration workflows using Azure Data Factory (ADF).
Language:TSQL 3shubhammirajkar / tokyo_olympics_de_project
Explore the Tokyo Olympics data journey! We ingested a GitHub CSV into Azure via Data Factory, stored it in Data Lake Storage Gen2, performed transformations in Databricks, conducted advanced analytics in Azure Synapse, and visualized insights in Synapse or Power BI.
Language:Jupyter Notebook 2- Language:PowerShell 1
- Language:Java 1
ayanhussain81 / Olympics-Data-ETL
Tokyo Olympic Azure Data Engineering Project
Language:Jupyter Notebook 1bennyaustin / ADFCustomActivity-AASPartition
Repository for Azure Data Factory (ADF) Custom Activity to dynamically create and process Azure Analysis Service (AAS) Tabular Model Partitions
Language:C# 1BrioGit / DataFactoryAutomation
This application shows how to use the Azure .NET library to create and execute Azure Data Factory from C# code.
Language:C# 1liliankotvan / Azure-ADF-RefreshPBI
Azure Data Factory template to refresh Power BI Dataset
MuhammadHasaanWahid / Data-Filtering-Pipeline-ETL
This Project Extracts supply chain data from csv file having 180k records and more than 40 columns from the Azure Datalake Gen2 storage account and do some dataanalysis with Python(Pandas) to find the top 3 countries and filtered the data for top 3 countries and finally transferred it to 3 files in datalake again by creating ETL pipeline in ADF.
Language:Jupyter Notebook 1snhaider9977 / Datafactory-CI-CD
This repository serves as a sample demonstration of implementing CI/CD practices on Azure Data Factory using Azure DevOps. It provides an example approach to configure a pipeline for continuous integration and continuous deployment (CI/CD) workflows on Azure Data Factory
tuancamtbtx / Azure-LandingZone-Demo
Azure Data Landing Zone - IAC , Config, Code
Language:HCL 1venkatakamaiah46 / Python
Interesting_programs_written_in_Python_language
ecaras / EmpowerID-Lab
This project showcase the following: 1. REST Connection to CHAT GPT 2. Logging system to Azure file system, storage app and insights. 3. Azure Data Factory Rest implementation using MS Graph client.
Language:C# 0hannah0wang / end-to-end-data-reporting
End to end data reporting project using Azure services like Azure Data Factory for data orchestration, Azure Synapse Analytics for data warehousing, Databricks for data transformations, and Power BI for intuitive data visualization and reporting.
Language:Jupyter Notebook 0luccayz / dataengineer_project_001
Efetuar o download de arquivos da web com Python. Inserir dados de um dataframe na cloud Azure com Azure SQL Database. Efetuar transformações nos dados com Azure Data Factory.
Language:Jupyter Notebook 0nazish555 / Tokyo-Olympics-Azure-Data-Engineering-Project
This project leverages Azure Cloud services like Azure Data Factory, Azure Databricks, and Synapse Analytics to execute a data engineering workflow. Utilizing data sourced from the Olympic API on GitHub, it involves extracting raw data into Azure Data Lake Storage, transforming it with PySpark on Azure Databricks, and analyzing the transformed data
Language:Jupyter Notebook 0abbgrade / PsDataFactory
PsDataFactory is a PowerShell module to generate DataFactory pipelines and datasets.
Language:PowerShell- Language:Python
epomatti / az-adf-s3-blob
Data integration with ADF
Language:HCLData engineering with Azure services
Language:HCLEthanBauer / adf
Repository to connect github to Azure Data Factory instance for CI/CD pipeline
Kokit0 / Tokyo-Olympics-Azure-Data-Eng-Project
Datas Engineer Exploration of the 2021 Tokyo Olympic data using a variety of tools and technologies, with a primary focus on Azure services, Leveraging Azure Data Factory, Data Lake Gen 2, Synapse Analytics, and Azure Databricks to perform data engineering tasks, analysis, and visualization.
Language:Jupyter Notebookmicvet / data-eng-project-amazon
O objetivo deste projeto foi aplicar os conhecimentos nas ferramentas de extração e tratamento de dados da plataforma Azure.
Language:Jupyter Notebooksanogotech / modern-data-warehouse-dataops
DataOps for the Modern Data Warehouse on Microsoft Azure. https://aka.ms/mdw-dataops.
Language:Shelltknishh / olympic-data-analysis-azure
End-to-End data engineering project with Azure Databricks as cloud service and Tokyo olympic data
Language:Jupyter Notebook
