PrefectHQ / prefect-dbt

Collection of Prefect integrations for working with dbt with your Prefect flows.

Home Page:https://prefecthq.github.io/prefect-dbt/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add smart retry flow

desertaxle opened this issue · comments

Create a flow that is capable of performing smart retries with dbt. @sungchun12 has an example of how to achieve this functionality here: https://github.com/sungchun12/prefect-dbt-cloud/blob/main/dbt_cloud_smart_rerun.py

Specifically the use cases:

Using this recipe will allow for intelligent retries in the following scenarios:

  • Connection timeouts with a database
  • Another dbt job is interacting with the same table as the current job and causing concurrency issues at the database level
  • Missing permissions to interact with tables/schemas
    - Requires manually updating permissions in the databse before restarting flow

Closed by #81