databrickslabs / dbx

🧱 Databricks CLI eXtensions - aka dbx is a CLI tool for development and advanced Databricks workflows management.

Home Page:https://dbx.readthedocs.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[docs] DLT example deployment file missing "default"

BGerwe opened this issue · comments

Expected Behavior

Using the DLT pipeline deployment file as a template, I should be able to fill in appropriate values and deploy the pipeline successfully.

Current Behavior

Trying to deploy the pipeline results in AttributeError: 'list' object has no attribute 'get'.

Steps to Reproduce (for bugs)

  1. Create a minimal deployment YAML based on the example deployment file. This looks something like:

conf/dlt_deployment.yml

environments:
  workflows:
    - name: "test_DLT_pipeline"
      workflow_type: "pipeline" 
      storage: "dbfs:/user/location"
      configuration:
        "property1": "value"
      clusters:
        - label: "some-label"
          autoscale:
            min_workers: 1
            max_workers: 2
            mode: "legacy" 
      libraries:
        - notebook:
            path: "/Repos/user/path_to_notebook"
      target: "some_target_db"
  1. Deploy the pipeline: dbx deploy --deployment-file conf/dlt_deployment.yml

Context

This is fixed by adding a default: level after environments:

environments:
  default:
    workflows:
      - name: "test_DLT_pipeline"
 ...

Your Environment

python 3.9.15

  • dbx version used: 0.8.14
  • Databricks Runtime version: 11.3 LTS ML