eriklangenborg-rs / dask-adlfs

Shim to allow Azure Datalake access in dask

Dask interface to Azure-Datalake Storage

Warning: this code is experimental and untested.

To install the backend, do import dask_adlfs, which will allow for URLs starting with "adl://" in dask functions.

Usage Notes:

From StackOverflow

import dask.dataframe as dd
df = dd.read_csv('adl://mystore/path/to/*.csv', storage_options={
    tenant_id='mytenant', client_id='myclient', 
    client_secret='mysecret'})