dpguthrie/snowflake-dbt-demo-new

Welcome to the dbt Labs demo dbt project! We use the TPCH dataset to create a sample project to emulate what a production project might look like!

                    _              __                   
   ____ ___  ____ _(_)___     ____/ /__  ____ ___  ____ 
  / __ `__ \/ __ `/ / __ \   / __  / _ \/ __ `__ \/ __ \
 / / / / / / /_/ / / / / /  / /_/ /  __/ / / / / / /_/ /
/_/ /_/ /_/\__,_/_/_/ /_/   \__,_/\___/_/ /_/ /_/\____/

Special demos

dbt-external-tables: Manage database objects that read data external to the warehouse within dbt. See models/demo_examples/external_sources.yml.
Lifecycle Notifications: See examples of dbt Cloud Job Lifecycle Notifications here.
Pivot tables: One example of creating a pivot table using Snowflake syntax, another example using Jinja. See models/aggregates/agg_yearly_*.sql.

dbt Cloud Attached CLI

The files in .devcontainer allow a user to spin up a container with the dbt Cloud CLI installed. The .devcontainer.json is currently minimally configured as a python container - feel free to modify or add features to fit your use case. You can do this either locally or using Github's Codespaces

Requirements

You'll need to set an environment variable, DBT_CLOUD_API_KEY. You can set this locally or when using within codespaces, you'll need to add a secret in Github - instructions here.

Codegen Examples

The codegen package can be run via the IDE, by clicking the "Compile" button, or in the command line.

Command Line

The example below shows how we can generate yml for a particular source:

dbt run-operation generate_source --args '{"schema_name": "tpch_sf001", "database_name": "raw", "generate_columns": "true", "include_descriptions": "true"}'

IDE

Paste in the snippets below in your IDE and click "Compile".

codegen.generate_source

Generates lightweight YAML for a Source

{{
    codegen.generate_source(
        schema_name='tpch_sf001',
        database_name='raw',
        generate_columns='true',
        include_descriptions='true',
    )
}}

codegen.generate_base_model

Generates SQL for a staging model

{{
    codegen.generate_base_model(
        source_name='tpch',
        table_name='orders',
    )
}}

codegen.generate_model_yaml

Generates the YAML for a given model

{{ 
    codegen.generate_model_yaml(
        model_name='stg_tpch_orders'
    )
}}

Generates the YAML for multiple models

{{ 
    generate_models_yaml(
        model_names=[
            'stg_tpch_orders',
            'stg_tpch_parts',
            'stg_tpch_regions',
        ]
    )
}}

dpguthrie / snowflake-dbt-demo-new