Empty source loading fails with `delta` table format on remote filesystems for not-yet-existing tables
jorritsandbrink opened this issue · comments
dlt version
0.5.3
Describe the problem
Empty source loading (e.g. when using dlt.mark.materialize_table_schema()) fails with delta table format on remote filesystems for not-yet-existing tables.
Reason is that storage_options are not passed to DeltaTable.create.
Does not fail for local filesystem because it does not need storage_options.
Expected behavior
No response
Steps to reproduce
Run test_delta_table_empty_source with remote filesystem (e.g. add AZ_BUCKET to bucket_subset).
Operating system
Windows
Runtime environment
Local
Python version
3.8
dlt data source
No response
dlt destination
No response
Other deployment details
No response
Additional information
No response