azure-data-lake

There are 1 repository under azure-data-lake topic.

• kahing / goofys

  a high-performance, POSIX-ish Amazon S3 file system written in Go

  s3-buckets3posixfilesystemaws-s3cloud-storagegolangfusegcsgoogle-cloud-storageopenstack-swiftfuse-filesystemazure-storageazure-blob-storageazure-blobazure-data-lake
  Language:Go 5046

• robinrodricks / FluentStorage

  A polycloud .NET cloud storage abstraction layer. Provides Blob storage (AWS S3, GCP, FTP, SFTP, Azure Blob/File/Event Hub/Data Lake) and Messaging (AWS SQS, Azure Queue/ServiceBus). Supports .NET 5+ and .NET Standard 2.0+. Pure C#.

  abstractionawsaws-s3aws-sqsazureazure-data-lakeazure-event-hubazure-queueazure-service-busblobblob-storagefilefile-storageftpftpsgcp
  Language:C# 203

• greenplum-db / pxf

  Platform Extension Framework: Federated Query Engine

  pxfhadoophdfss3query-federationgreenplumhbasegoogle-cloud-storageazure-blob-storageazure-data-lakehivegreenplum-extension
  Language:Java 78

• Azure / AzureStor

  R interface to Azure storage accounts

  razure-storageazure-storage-blobazure-storage-fileazure-data-lakeazure-sdk-r
  Language:R 61

• ashwin-patil / threat-hunting-with-notebooks

  Repository with Sample threat hunting notebooks on Security Event Log Data Sources

  threat-huntingjupyter-notebookazure-data-lakeazure-data-explorerrpythonanacondasecuritydatasciencewef
  Language:Jupyter Notebook 58

• justBlindbaek / TraditionalModernDW

  Simple cloud only DWH solution architecture.

  architecturepowerbiazure-sql-databaseazure-data-factoryazure-data-lake
  Language:TSQL 38

• cloudyr / AzureStor

  Interface to Azure storage accounts. Submit issues and PRs at https://github.com/Azure/AzureStor

  azure-data-lakeazure-storageazure-storage-blobazure-storage-filer
  Language:R 22

• syedhassaanahmed / databricks-notebooks

  Collection of Databricks and Jupyter Notebooks

  sparkazure-databrickspysparkazure-event-hubskafkacosmos-dbspark-sqlparquetpower-biazure-storageazure-data-lakejupyter-notebooksmatplotlibazure-iothubgraphframeshive-udfpandas-dataframeazure-sql-databasemongodbspark-udf
  Language:Jupyter Notebook 22

• arsenvlad / docker-presto-adls-wasb

  Example of a single node Presto with Azure Data Lake Store (ADLS) and Azure Storage Blob (WASB) access via Hive metastore

  dockerprestoazure-data-lakeazure-blob-storage
  Language:Dockerfile 18

• Jayvardhan-Reddy / Azure-Certification-DP-201

  Road to Azure Data Engineer Part-II: DP-201 - Designing an Azure Data Solution

  azureazure-certificationazure-certsresourcesdp-201data-engineeringdata-engineermicrosoft-azuredesignazure-storageazure-securityazure-data-solutionazure-active-directoryazure-data-factoryazure-data-lakeazure-data-warehouseazure-cosmos-dbazure-stream-analyticsbatch-processingstream-processing
  18

• AnthonyByansi / Azure-Data-Fundamentals-Guide

  A comprehensive guide to understanding and implementing data management and analytics solutions in the Azure ecosystem using Azure Data Fundamentals.

  azureazure-active-directoryazure-data-factoryazure-data-lakeazure-data-studioazure-databaseazure-databricksazure-synapse-analytics
  Language:Mermaid 16
• SAP_AND_COMMON_DATA_MODEL_DEMO

  ROBROICH / SAP_AND_COMMON_DATA_MODEL_DEMO

  This demo describes the basic integration between S/4HANA and the Microsoft Common Data Model (Model)

  azure-data-lake4hanadata-extraction4hana-semanticscdm-model
  16

• segovoni / azure-data-lake-store-delphi

  Microsoft Azure Data Lake Store Library for Delphi

  delphi-librarydelphiazure-data-lakerest-client
  Language:Pascal 14

• xpertdev / tdameritrade-streaming

  Streaming order book data using TD Ameritrade API

  real-timeazure-data-lakewebsocketorderbooklevel1level2tdameritrade-apitdameritradetimesaletda-apipython
  Language:Python 11

• dataplat / AzureDataPipelineTools

  A collection of Azure Function to make building Azure Data Factory pipeline simpler and easier.

  azureazure-data-factoryazure-data-lakeazure-functionsdata-engineering
  Language:C# 10

• amynic / TechHer

  Repo containing files for TechHer event and 'Let your Data tell you the Real Story: Advanced Analytics on Azure' hands on lab

  azureazure-data-lakeazure-machine-learningbigdatacognitive-servicesrserversimpsons
  9

• AdamPaternostro / Azure-Big-Data-and-Machine-Learning-Architecture

  A ready to use architecture for processing data and performing machine learning in Azure

  azurebig-dataazure-data-lakeazure-data-factoryml-ops
  Language:C# 8

• Jcardif / SerengetiDataLab

  An E2E solution of the Data Resources on Azure using the Snapshot Serengeti dataset. This E2E solution focuses Azure Synapse Analytics, Power Bi & the Azure Data Factory.

  azure-dataazure-data-factoryazure-data-lakeazure-mlazure-sqlazure-storageazure-synapse-analyticsmicrosoft-purviewpower-bisynapse-analytics
  Language:Bicep 8

• jksinghpro / kafka-connect-adl

  Kafka Connect Connector for ADLS(Azure Data Lake Store)

  kafka-connectkafkaazure-data-lake
  Language:Java 7

• oleewere / fluent-plugin-azurestorage-gen2

  Fluentd output plugin for Azure Datalake Storage Gen2 (append support)

  azure-data-lakefluentdfluent-pluginazure-containerazure-oauthazure-storageadlsgen2adlsfluentd-plugin
  Language:Ruby 7

• tomkerkhove / gdpr-with-azure

  Scenarios on how you can be GDPR compliant by using Azure services

  azureazure-data-factoryazure-data-lakeazure-logicapps
  Language:C# 6

• syedhassaanahmed / azure-kafka-spark-adls

  Azure ARM template to deploy Kafka and Spark clusters in same VNet with ADLS

  kafkahdinsightvnetarm-templatesazure-data-lakecertificatebash-scriptopensslservice-principal
  Language:Shell 5

• Watts-Energy / Watts.Azure

  A collection of utilities for working with Azure Batch, Azure Data Factory, Azure Table Storage and Azure Blob Storage.

  azureazure-table-storageazure-blobazure-batchrazure-data-factoryazure-data-lakeazure-storagecsharpdotnet-frameworkazure-service-bus
  Language:C# 4

• semashkinvg / Azure.HowTos

  azureazure-data-lakecsharpnet-coreazure-sdk
  Language:C# 3

• sindhudweep / Orcneas

  Read and Extract from ORC files for U-SQL

  azure-data-lakeu-sqlorc
  Language:C# 3

• AdamPaternostro / Azure-Databricks-External-Hive-and-ADLS

  Shows how to use an External Hive (SQL Server) along with ADLS Gen 1 as part of a Databricks initialization script that runs when the cluster is created.

  azureazure-data-lakeazure-databrickshive-metastore
  Language:PowerShell 2

• ArtisConsulting / customer-insights-azure-data-workshop

  Workshop for integrating Dynamics 365 Customer Insights and Azure Data Services

  customer-insightsazure-synapse-analyticsazure-synapsemapping-dataflowsazure-machine-learningazure-machine-learning-studioazure-data-lakedynamics-365-customer-insightsarm-templatearm-templatesazure-arm-templatesazure-arm-templatesynapse-studioazure-synapse-serverless-sqlworkshophands-on-lab
  2

• AdamPaternostro / Azure-HDI-DistCP

  Creates a HDInsight cluster then runs distcp remotely to copy data between blob and/or data lake (ADLS)

  azurehdinsightdistcphadoopfile-copyazure-storageazure-data-lake
  Language:Shell 1

• AdamPaternostro / Azure-HDInsight-ARM-Template

  Creates an HDInsight cluster that has an external Hive metastore and access to Azure Data Lake Store

  hdinsighthdinsight-service-principleazure-arm-templatesazure-data-lake
  1

• amperity / blocks-adl

  Content-addressable Azure Data Lake block store

  azure-data-lakeclojurecontent-addressable-storage
  Language:Clojure 1

• Data-Culpa / dataculpa-azure-datalake-gen2

  Azure Data Lake Gen2 storage connectors for Data Culpa - monitor data quality automatically with Data Culpa Validator

  azure-data-lakedataculpadataquality
  Language:Python 1

• haxxorsid / flink-datalake-bulk-upload

  Bulk image streaming and upload using Flink (+ Kubernetes), Kafka, Data Lake, and SQL (Provided with React UI and Node server for Demo).

  flink-stream-processingflinkkafkaazureazure-event-hubazure-data-lakenodejsreactkubernetesdockersqlazure-sql-database
  Language:Java 1

• poojatripathi06 / Covid-reporting-adf

  building a real-world data pipeline in Azure Data Factory (ADF) dataset provided by https://www.ecdc.europa.eu/ ingesting data from sources such as HTTP and Azure Blob Storage into Azure Data Lake Gen2 using ADF. transformed data and loaded transformed data using Databricks Notebook Activity in Azure Data Factory (ADF) and load into Azure Data Lake Storage Gen2.

  azure-data-factoryazure-data-lakeazure-databricksazure-pipelinesazure-sql-databaseblob-storagepowerbi
  1
• azure-databricks-etl-project

  randyroac / azure-databricks-etl-project

  ETL motor racing data project using Azure Databricks, Pyspark and Azure Date Lakes

  azure-data-lakeazure-databricks
  Language:Python 1

• venkatakamaiah46 / Azure

  POC projects working on Cloud Platforms

  adlsgen2azuredatabrickspysparkpythonsqlazure-data-factoryazure-data-lakeazure-databricksazure-devopsazure-pipelinesazure-storageazure-synapse-analyticsazure-synapse-dwhblob-storagesnowflake-data-warehouse
  Language:HTML 1

• vrup0408 / IPL-Data-Analytics

  We have dataset of IPL from 2008 to 2020 and we have to visualize analytics on Power BI dashboard. We have to upload that dataset into data lake. After that we have to process that data through pipeline and produce modeled data in warehouse. So, that we will be able to analyze the data in Power BI through pre-defined dashboards.

  azureazure-data-factoryazure-data-lakedashboarddata-warehouseiplpowerbisnowflakesnowpark
  1