StrezlessMusick / dbt_ad_reporting

Fivetran's ad reporting dbt package. Combine your Facebook, Google, Pinterest, Linkedin, Twitter, Snapchat and Microsoft advertising spend using this package.

Home Page:https://fivetran.github.io/dbt_ad_reporting/#!/overview

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Ad Reporting dbt Package (Docs)

📣 What does this dbt package do?

NOTE: You do not need to have all of these connector types to use this package, though you should have at least two.

  • Generates a comprehensive data dictionary of your source and modeled App Reporting data via the dbt docs site

Refer to the table below for a detailed view of final models materialized by default within this package. Additionally, check out our Docs site for more details about these models.

model description
ad_reporting__account_report Each record represents daily metrics by account
ad_reporting__campaign_report Each record represents daily metrics by campaign and account.
ad_reporting__ad_group_report Each record represents daily metrics by ad group, campaign and account.
ad_reporting__ad_report Each record represents daily metrics by ad, ad group, campaign and account.
ad_reporting__keyword_report Each record represents daily metrics by keyword, ad group, campaign and account.
ad_reporting__search_report Each record represents daily metrics by search query, ad group, campaign and account.
ad_reporting__url_report Each record represents daily metrics by URL (and if applicable, URL UTM parameters), ad group, campaign and account.

The individual platform models may have additional platform-specific metrics and fields better suited for deep-dive analyses at the platform level.

🎯 How do I use the dbt package?

Step 1: Pre-Requisites

  • Connector: Have all relevant Fivetran app platform connectors syncing data into your warehouse. This package currently supports:
  • Database support: This package has been tested on BigQuery, Snowflake, Redshift, Postgres and Databricks. Ensure you are using one of these supported databases.
  • dbt Version: This dbt package requires you have a functional dbt project that utilizes a dbt version within the respective range >=1.0.0, <2.0.0.
  • For Facebook Ads compatibility, please ensure that you have configured the necessary pre-built reports. Please refer to the Facebook Ads package for more information.

Step 2: Installing the Package

Include the following github package version in your packages.yml

Check dbt Hub for the latest installation instructions, or read the dbt docs for more information on installing packages.

packages:
  - package: fivetran/ad_reporting
    version: [">=1.0.0", "<1.1.0"]

Step 3: Configure Database and Schema Variables

By default, this package looks for your ad platform data in your target database. If this is not where your app platform data is stored, add the relevant <connector>_database variables to your dbt_project.yml file (see below).

vars:
    apple_search_ads_schema: apple_search_ads
    apple_search_ads_database: your_database_name

    facebook_ads_schema: facebook_ads
    facebook_ads_database: your_database_name 

    google_ads_schema: google_ads
    google_ads_database: your_database_name 

    microsoft_ads_schema: bingads
    microsoft_ads_database: your_database_name

    linkedin_ads_schema: linkedin_ads 
    linkedin_ads_database: your_database_name  

    pinterest_schema: pinterest
    pinterest_database: your_database_name 

    twitter_ads_schema: twitter_ads
    twitter_ads_database: your_database_name  

    snapchat_schema: snapchat_ads
    snapchat_database: your_database_name 
    
    tiktok_ads_schema: tiktok_ads
    tiktok_ads_database: your_database_name 

Step 4: Enabling/Disabling Models

This package takes into consideration that not every account will have every feature enabled per platform. If your syncs exclude certain tables, it is because you either don't use that functionality in your respective ad platforms or have actively excluded some tables from your syncs.

Disable Platform Specific Reporting

If you would like to disable all reporting for any specific platform, please include the respective variable(s) in your dbt_project.yml.

vars:
  ad_reporting__apple_search_ads_enabled: False # by default this is assumed to be True
  ad_reporting__pinterest_ads_enabled: False # by default this is assumed to be True
  ad_reporting__microsoft_ads_enabled: False # by default this is assumed to be True
  ad_reporting__linkedin_ads_enabled: False # by default this is assumed to be True
  ad_reporting__google_ads_enabled: False # by default this is assumed to be True
  ad_reporting__twitter_ads_enabled: False # by default this is assumed to be True
  ad_reporting__facebook_ads_enabled: False # by default this is assumed to be True
  ad_reporting__snapchat_ads_enabled: False # by default this is assumed to be True
  ad_reporting__tiktok_ads_enabled: False # by default this is assumed to be True

Enable/Disable Specific Reports within Platforms

For Apple Search Ads, if you are not utilizing the search functionality, you may choose to update the respective variable below.

For Twitter Ads, if you are tracking keyword performance, you may choose to update the corresponding variable below.

Add the following variables to your dbt_project.yml file

vars:
  apple_search_ads__using_search_terms: False # by default this is assumed to be True
  twitter_ads__using_keywords: False # by default this is assumed to be True

(Recommended) Step 5: Change the Build Schema

By default this package will build all models in your <target_schema> with the respective package suffixes (see below). This behavior can be tailored to your preference by making use of custom schemas. If you would like to override the current naming conventions, please add the following configuration to your dbt_project.yml file and rename +schema configs:

models:  
  ad_reporting:
    +schema: ad_reporting

  apple_search_ads:
    +schema: apple_search_ads
  apple_search_ads_source:
    +schema: apple_search_ads_source
  
  facebook_ads:
    +schema: facebook_ads
  facebook_ads_source:
    +schema: facebook_ads_source
  
  google_ads:
    +schema: google_ads
  google_ads_source:
    +schema: google_ads_source

  linkedin:
    +schema: linkedin
  linkedin_source:
    +schema: linkedin_source

  microsoft_ads:
    +schema: microsoft_ads
  microsoft_ads_source:
    +schema: microsoft_ads_source

  pinterest:
    +schema: pinterest
  pinterest_source:
    +schema: pinterest_source
  
  twitter_ads:
    +schema: twitter_ads
  twitter_ads_source:
    +schema: twitter_ads_source
  
  snapchat_ads:
    +schema: snapchat_ads
  snapchat_ads_source:
    +schema: snapchat_ads_source
  
  tiktok_ads:
    +schema: tiktok_ads
  tiktok_ads_source:
    +schema: tiktok_ads_source

Provide a blank +schema: to write to the target_schema without any suffix.

(Optional) Step 6: Additional configurations

Expand for details

Change the source table references

If an individual source table has a different name than the package expects, add the table name as it appears in your destination to the respective variable:

IMPORTANT: See the Apple Store dbt_project.yml and Google Play dbt_project.yml variable declarations to see the expected names.

vars:
    <default_source_table_name>_identifier: your_table_name 

(Optional) Step 7: Orchestrate your models with Fivetran Transformations for dbt Core™

Expand for details

Fivetran offers the ability for you to orchestrate your dbt project through Fivetran Transformations for dbt Core™. Learn how to set up your project for orchestration through Fivetran in our Transformations for dbt Core™ setup guides.


(Optional) Step 8: Use predefined Metrics

Expand for details

On top of the ad_reporting__ad_report final model, the Ad Reporting dbt package defines common Metrics, including:

  • Spend
  • Impressions
  • Clicks
  • Cost per click
  • Clickthrough rate
  • Bounce rate
  • Count of active ads
  • Average spend
  • Average non-zero spend

You can find the supported dimensions and full definitions of these metrics here.

To use dbt Metrics, add the dbt metrics package to your project's packages.yml file:

packages:
  - package: dbt-labs/metrics
    version: [">=0.3.0", "<0.4.0"]

Note: The Metrics package has stricter dbt version requirements. As of today, the latest version of Metrics (v0.3.5) requires dbt [">=1.2.0-a1", "<2.0.0"].

To utilize the Ad Reporting's pre-defined metrics in your code, refer to the dbt metrics package usage instructions and the example below:

select *
from {{ metrics.calculate(
        metric('clicks'),
        grain='month',
        dimensions=['platform', 
                    'campaign_id', 
                    'campaign_name'
        ],
        secondary_calculations=[
            metrics.period_over_period(comparison_strategy='difference', interval=1, alias='diff_last_mth'),
            metrics.period_over_period(comparison_strategy='ratio', interval=1, alias='ratio_last_mth')
        ]
) }}

🔍 Does this package have dependencies?

This dbt package is dependent on the following dbt packages. For more information on the below packages, refer to the dbt hub site.

If you have any of these dependent packages in your own packages.yml I highly recommend you remove them to ensure there are no package version conflicts.

packages: 
  - package: fivetran/fivetran_utils
    version: [">=0.3.0", "<0.4.0"]

  - package: dbt-labs/dbt_utils
    version: [">=0.8.0", "<0.9.0"]

  - package: calogica/dbt_expectations
    version: [">=0.5.0", "<0.6.0"]

  - package: fivetran/apple_search_ads
    version: [">=0.1.0", "<0.2.0"]

  - package: fivetran/apple_search_ads_source
    version: [">=0.1.0", "<0.2.0"]
  
  - package: fivetran/facebook_ads
    version: [">=0.5.0", "<0.6.0"]

  - package: fivetran/facebook_ads_source
    version: [">=0.5.0", "<0.6.0"]
  
  - package: fivetran/google_ads
    version: [">=0.8.0", "<0.9.0"]

  - package: fivetran/google_ads_source
    version: [">=0.8.0", "<0.9.0"]

  - package: fivetran/pinterest
    version: [">=0.6.0", "<0.7.0"]

  - package: fivetran/pinterest_source
    version: [">=0.6.0", "<0.7.0"]

  - package: fivetran/microsoft_ads
    version: [">=0.5.0", "<0.6.0"]

  - package: fivetran/microsoft_ads_source
    version: [">=0.6.0", "<0.7.0"]

  - package: fivetran/linkedin
    version: [">=0.5.0", "<0.6.0"]

  - package: fivetran/linkedin_source
    version: [">=0.5.0", "<0.6.0"]

  - package: fivetran/twitter_ads
    version: [">=0.5.0", "<0.6.0"]

  - package: fivetran/twitter_ads_source
    version: [">=0.5.0", "<0.6.0"]

  - package: fivetran/snapchat_ads
    version: [">=0.4.0", "<0.5.0"]

  - package: fivetran/snapchat_ads_source
    version: [">=0.4.0", "<0.5.0"]

  - package: fivetran/tiktok_ads
    version: [">=0.2.0", "<0.3.0"]

  - package: fivetran/tiktok_ads_source
    version: [">=0.2.0", "<0.3.0"]

🙌 How is this package maintained and can I contribute?

Package Maintenance

The Fivetran team maintaining this package only maintains the latest version of the package. We highly recommend you stay consistent with the latest version of the package and refer to the CHANGELOG and release notes for more information on changes across versions.

Opinionated Decisions

In creating this package, which is meant for a wide range of use cases, we had to take opinionated stances on a few different questions we came across during development. We've consolidated significant choices we made in the DECISIONLOG.md, and will continue to update as the package evolves. We are always open to and encourage feedback on these choices, and the package in general.

Contributions

These dbt packages are developed by a small team of analytics engineers at Fivetran. However, the packages are made better by community contributions!

We highly encourage and welcome contributions to this package. Check out this post on the best workflow for contributing to a package!

🏪 Are there any resources available?

  • If you encounter any questions or want to reach out for help, please refer to the GitHub Issue section to find the right avenue of support for you.
  • If you would like to provide feedback to the dbt package team at Fivetran, or would like to request a future dbt package to be developed, then feel free to fill out our Feedback Form.
  • Have questions or want to just say hi? Book a time during our office hours here or send us an email at solutions@fivetran.com.

About

Fivetran's ad reporting dbt package. Combine your Facebook, Google, Pinterest, Linkedin, Twitter, Snapchat and Microsoft advertising spend using this package.

https://fivetran.github.io/dbt_ad_reporting/#!/overview