adlsgen2

There are 1 repository under adlsgen2 topic.

• procter-gamble-oss / octopufs

  OctopuFS library helps managing cloud storage, ADLSgen2 specifically. It allows you to operate on files (moving, copying, setting ACLs) in very efficient manner. Designed to work on databricks, but should work on any other platform as well.

  azure-storagedatabricksadlsgen2hadoop-filesystemsparkscala
  Language:Scala 11

• gerardwolf / blog

  Repository for all blog scripts and code

  deltalakedatabricksadlsgen2
  Language:TSQL 7

• jlsilva01 / adls-azure

  Procedimento para criação de um Azure Data Lake Storage usando Terraform, através de uma assinatura MS Learn Sandbox

  adlsgen2azuredata-laketerraformazurecli
  Language:HCL 7

• oleewere / fluent-plugin-azurestorage-gen2

  Fluentd output plugin for Azure Datalake Storage Gen2 (append support)

  azure-data-lakefluentdfluent-pluginazure-containerazure-oauthazure-storageadlsgen2adlsfluentd-plugin
  Language:Ruby 7

• paolosalvatori / blob-private-endpoint

  This sample demonstrates how to create a Linux Virtual Machine in a virtual network that privately accesses a blob storage account using an Azure Private Endpoint.

  blob-storageazure-storageadlsgen2azure-storage-blobvirtual-machinevirtual-networkazure-virtual-machineazure-virtual-networksprivate-linkprivate-dns-zoneprivate-endpoint
  Language:Shell 3

• easonlai / sas_access_to_adls_databricks

  Using SAS to authenticate and access to ADLS Gen 2 from Azure Databricks

  adlsgen2azureazuredatabricksdatabricksblobstorageblob-storageblob-storage-accountshared-access-signaturesparkdata-analyticsdata-analysis-python
  Language:Jupyter Notebook 1

• iBalajiShanmugam / covid19-adf

  COVID19-ADF is a project that leverages Azure services to collect, analyze, and visualize COVID-19 data. With seamless data integration and advanced analytics, it provides valuable insights into the pandemic's impact, enabling informed decision-making in the fight against COVID-19.

  adfadlsgen2azurecovid-19data-pipelineecdchdinsightpipelinepowerbisql
  1

• sankamuk / ADLSGen2Admin

  Azure ADLS Gen2 CLI Tool

  adlsgen2powershellbashazurefilesystem
  Language:PowerShell 1

• shubhammirajkar / tokyo_olympics_de_project

  Explore the Tokyo Olympics data journey! We ingested a GitHub CSV into Azure via Data Factory, stored it in Data Lake Storage Gen2, performed transformations in Databricks, conducted advanced analytics in Azure Synapse, and visualized insights in Synapse or Power BI.

  adlsgen2azuresynapsedatabricksdatafactory
  Language:Jupyter Notebook 1

• sumeghasetia / azure-dataplatform-setup

  Implementation of most useful services of Azure Data Platform.

  azurecosmosdbazure-functionsazure-pipelinesazure-storageadlsgen2azure-sql-databaseazure-sql-dbazure-data-factorypolybasepostazure-postgresazure-mariadb
  Language:TSQL 1

• venkatakamaiah46 / Azure

  POC projects working on Cloud Platforms

  adlsgen2azureazure-data-factoryazure-data-lakeazure-databricksazure-devopsazure-pipelinesazure-storageazure-synapse-analyticsazure-synapse-dwhblob-storagedatabrickspysparkpythonsnowflake-data-warehousesql
  Language:HTML 1

• ayush9892 / Supply-Chain-ETL

  Data Engineering Project on Supply Chain ETL. Creating a dynamic ADF pipeline to ingest both Full Load and Incremental Load data from SQL Server and then transform these datasets based on medallion architecture using Databricks.

  adf-pipelineadlsgen2azureazurekeyvaultdatabricksextract-transform-loadpysparksql-server
  Language:Jupyter Notebook 0

• ayush9892 / SynapseSQLPool-DynamicView

  Creating a pipeline that will automatically create View of data in Synapse, whenever data arrives in ADLS Gen2.

  azuredynamicpipelinesynapse-analyticsadlsgen2t-sql
  0

• iBalajiShanmugam / formual1

  "Explore Formula 1 data analytics with this project. Leveraging the Ergast API, it utilizes Databricks Spark for ingestion, transformation, and analysis. ADLS acts as the storage layer, while Power BI visualizes the ADLS presentation layer. Uncover insights in the world of Formula 1 through powerful data analytics."

  adlsgen2azure-data-factoryazure-databricksdatabricksdatalakedelta-lakeergast-apiformula1powerbi
  Language:Python 0

• anideswandikar1 / DataLakeUsageReport

  Code/Utility to recursively traverse a given Azure Data Lake Gen2 account and find the size of various Containers and Folders

  adlsgen2containersizeusagereportfolderscontainerssize
  Language:PowerShell

• anshul-cached / sync-adls

  Azure Data Lake Gen2 Backup Sync

  azurefunctionsevent-gridadlsgen2snapshodata-sync
  Language:Python

• bijoychaudhury / spark_aggregation_framework

  This repo contains code specific to the SQL-driven spark aggregation framework to be executed in the Databricks cluster that integrates with the Azure storage account.

  azuredatabrickssparkspark-sqladlsgen2cloud
  Language:Scala

• epomatti / az-datalake

  Azure Data Lake Gen2 with azcopy

  adlsgen2azureazure-data-laketerraform
  Language:HCL

• just-modeling / jupyterhub-k8s-apache-spark

  Deploy apache spark in client mode on Kubernetes cluster, integrate with Jupyter notebook through Jupyterhub server.

  apache-sparkautoscalingazuredelta-lakejupyterjupyter-notebookjupyterhubjupyterlabk8skubernetesnodepoolspark-uitaints-tolerationsadlsgen2azure-data-lakeazure-databricks
  Language:Shell