Giters

Stevedow99 / dbt-cloud-cancel-running-ci-job-action

This actions checks if a dbt cloud CI job is running after a new or updated PR is made. If there is a job running it will cancel it in order to kick of the new CI job

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

dbtgithub-actions

dbt Cloud Cancel Running CI Job Runs Action

This action checks if a dbt Cloud CI job (based on job ID) is running after a new or updated PR commit is made. If there is a job running or there are job runs that are queued - this action will cancel these running and queued runs since they are likely stale as a new commit to the PR has been made. This allows to always have the newest code commit running in the CI job without having to wait for the stale job runs to finish.

Some scenarios where this is useful:

  • You have longer running CI jobs in your dbt Cloud project and you don't want new commits to get queued when they are replacing stale code that is still running in a previous CI job
  • You have a large team where there is constant commits/PRs against the code base and you always want the latest job to be run without waiting for older jobs to finish

To view a Loom video of this action being used, click here

Inputs

Required Inputs

  • dbt_cloud_token - dbt Cloud API token
  • dbt_cloud_account_id - dbt Cloud Account ID
  • dbt_cloud_job_id - dbt Cloud Job ID

Optional Inputs

  • only_cancel_run_if_commit_is_using_pr_branch - A flag that can be set to true or false
    • When this flag is set to true, CI jobs will only get cancelled if the branch that the CI job is running off of is the same as the branch the PR is using
    • Example of using this: If someone kicks off a CI run based off the branch branch_1 and then someone else kicks off a CI run based on the branch branch_2
      • When this flag is set to true the GitHub Action triggered via the PR on branch_2 will not cancel the CI run based on branch_1
      • When this flag is set to false (or not set at all) the GitHub Action will cancel the run based on branch_1 as it doesn't care if the branch is related to the newest PR or not - the behavior is to always get the lastest CI job running regardless of branch
    • When this flag is set to true it requires that the input github_pr_number be configured to ${{ github.event.number }}
  • github_pr_number - The number of the pull request in GitHub, this is used when limiting cancelling of jobs to just a given PR that is being worked on This is only needed if only_cancel_run_if_commit_is_using_pr_branch is set to true
  • dbt_cloud_host - the URL of the dbt cloud account with, by default cloud.getdbt.com is used
  • max_runs - the number of runs to look back and cancel for a given dbt Cloud job, by default 10 is used
  • only_cancel_queued_starting_run - A flag that can be set to true or false
    • When this flag is set to true, the action will only cancel dbt Cloud job runs that are in the state Queued or Starting
    • The purpose of this flag is to prevent the action from cancelling jobs that are already running when a new CI job is kicked off. Some use cases require this as they don't want to cancel a job that is halfway completed
  • cancel_runs_based_on_schema_override - A flag that can be set to true or false
    • When this and only_cancel_run_if_commit_is_using_pr_branch are both set to true, this action will find the associated PR number of existing dbt runs based on their schema_override. Specifically it assumes the schema_override will be of the form dbt_cloud_pr_{dbt_cloud_job_id}_{github_pr_number}, which matches that of dbt Cloud CI.
    • The purpose of this flag is to enable canceling stale CI runs even if they weren't triggered directly by the dbt Cloud <> GitHub integration. For example, teams triggering CI jobs via GitHub Actions (or any other CI/CD tool) can still utilize this action, as long as the schema_override parameter is configured properly in the CI job.

It's recommend to pass sensitive variables as GitHub secrets. Example article on how to use Github Action secrets

Outputs

  • cancelled_jobs_flag - A returned flag that is outputted as True if running/queued job runs were cancelled in order to kick off a CI job for the latest commit. Returns False if no job runs were running/queued and therefore didn't need to be cancelled.
  • cancelled_dbt_cloud_job_runs - A list of dbt Cloud run IDs for the given job that were cancelled in order to kick off a CI job for the latest commit. (e.g. [85660002, 85660002]). This is useful for logging.
  • cancelled_dbt_cloud_job_runs_markdown - Pre-Scripted markdown containing info on the cancelled jobs, can be used in PR comments. Example of a scenario where 4 job runs where cancelled this is the markdown output: 
    **The following dbt Cloud job runs were cancelled to free up the queue for the new CI job on the current PR:**
- Run **85519539** was cancelled at **2022-09-29 23:32:44 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519539/)
- Run **85519497** was cancelled at **2022-09-29 23:32:45 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519497/)
- Run **85519494** was cancelled at **2022-09-29 23:32:46 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519494/)
- Run **85519490** was cancelled at **2022-09-29 23:32:46 UTC**, view this run in dbt Cloud [here](https://cloud.getdbt.com/next/deploy/12345/projects/161955/runs/85519490/)

Creating a workflow

A workflow using the only_cancel_run_if_commit_is_using_pr_branch flag

# This is a basic workflow to show using this action

# name of the workflow
name: Cancel running slim ci job runs if new commit is made to the same pull request triggering a new run

# Controls when the workflow will run
on:
  pull_request:
    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab if needed
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

  # This workflow contains a single job called "cancel_running_slim_ci_jobs"
  cancel_running_slim_ci_jobs:
  
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:

      # running the step to cancel another other CI job runs that are running except the latest, only for the given PR
      - name: Cancel other CI runs that are running for the given PR except the latest run
        id: cancel_stale_ci_runs
        uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
        with:
          dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
          dbt_cloud_account_id: 12345
          dbt_cloud_job_id: 130247
          only_cancel_run_if_commit_is_using_pr_branch: true
          github_pr_number: ${{ github.event.number }}
          max_runs: "50"

A workflow not using the only_cancel_run_if_commit_is_using_pr_branch flag

# This is a basic workflow to show using this action

# name of the workflow
name: Cancel all other running slim ci job runs if new commit is made

# Controls when the workflow will run
on:
  pull_request:
    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab if needed
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

  # This workflow contains a single job called "cancel_running_slim_ci_jobs"
  cancel_running_slim_ci_jobs:
  
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:

      # running the step to cancel another other CI job runs that are running except the latest
      - name: Cancel all other ci runs that are running except the latest run
        id: cancel_stale_ci_runs
        uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
        with:
          dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
          dbt_cloud_account_id: 12345
          dbt_cloud_job_id: 130247

Examples of using this Github action with other workflows


Example using workflow to log the cancelled job run IDs:

# This is a basic workflow to show using this action

# name of the workflow
name: Cancel running slim ci job runs if new commit is made to the same pull request triggering a new run

# Controls when the workflow will run
on:
  pull_request:
    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab if needed
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

  # This workflow contains a single job called "cancel_running_slim_ci_jobs"
  cancel_running_slim_ci_jobs:
  
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:

      # running the step to cancel another other CI job runs that are running except the latest, only for the given PR
      - name: Cancel other CI runs that are running for the given PR except the latest run
        id: cancel_stale_ci_runs
        uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
        with:
          dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
          dbt_cloud_account_id: 12345
          dbt_cloud_job_id: 130247
          only_cancel_run_if_commit_is_using_pr_branch: true
          github_pr_number: ${{ github.event.number }}
          max_runs: "50"
          
      # logging if there was a job run(s) cancelled or not
      - name: Logging if there was a CI run that was cancelled
        if:  steps.cancel_stale_ci_runs.outputs.cancelled_jobs_flag == 'True'
        run: |
          echo "A dbt Cloud CI job run(s) was cancelled due to a new CI run being triggered"
          echo "The following job runs were cancelled ${{ steps.cancel_stale_ci_runs.outputs.cancelled_dbt_cloud_job_runs }}"

Example using workflow to cancel job runs and then post information about the cancelled job runs in a PR comment

This workflow will produce a PR comment that looks like this when job runs are cancelled:

image

# This is a basic workflow to show using this action

# name of the workflow
name: Cancel running slim ci job runs if new commit is made to the same pull request triggering a new run

# Controls when the workflow will run
on:
  pull_request:
    branches: [ "main" ]

  # Allows you to run this workflow manually from the Actions tab if needed
  workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

  # This workflow contains a single job called "cancel_running_slim_ci_jobs"
  cancel_running_slim_ci_jobs:
  
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:

      # running the step to cancel another other CI job runs that are running except the latest, only for the given PR
      - name: Cancel other CI runs that are running for the given PR except the latest run
        id: cancel_stale_ci_runs
        uses: Stevedow99/dbt-cloud-cancel-running-ci-job-action@v1.3
        with:
          dbt_cloud_token: ${{ secrets.DBT_CLOUD_TOKEN }}
          dbt_cloud_account_id: 12345
          dbt_cloud_job_id: 130247
          only_cancel_run_if_commit_is_using_pr_branch: true
          github_pr_number: ${{ github.event.number }}
          max_runs: "50"
          
      # logging if there was a job run(s) cancelled or not
      - name: Logging if there was a CI run that was cancelled
        if:  steps.cancel_stale_ci_runs.outputs.cancelled_jobs_flag == 'True'
        run: |
          echo "A dbt Cloud CI job run(s) was cancelled due to a new CI run being triggered"
          echo "The following job runs were cancelled ${{ steps.cancel_stale_ci_runs.outputs.cancelled_dbt_cloud_job_runs }}"
          
      # if there a job run(s) was cancelled, we grab the outputted markdown and put it into a PR comment for logging
      - name: PR comment with Job Run Cancelation Information
        uses: mshick/add-pr-comment@v1
        if:  steps.cancel_stale_ci_runs.outputs.cancelled_jobs_flag == 'True'
        with:
          message: |
            ${{ steps.cancel_stale_ci_runs.outputs.cancelled_dbt_cloud_job_runs_markdown }}
          repo-token: ${{ secrets.GITHUB_TOKEN }}
          repo-token-user-login: 'github-actions[bot]'
          allow-repeats: false # This is the default

About

This actions checks if a dbt cloud CI job is running after a new or updated PR is made. If there is a job running it will cancel it in order to kick of the new CI job

dbtgithub-actions

License:MIT License

Languages

Language:Python 96.2%Language:Dockerfile 3.8%