testing-farm-as-github-action

Testing Farm as GitHub Action is a GitHub Action for executing tests on the Testing Farm Service.

The tests to run are to be described with a tmt plan by the user of this GitHub Action. Pull Request status is automatically updated after the tests are executed, if this option is enabled with the update_pull_request_status user-defined input variable.

The Action uses ubuntu-20.04 and it is a composite action. It internally downloads needed binaries curl and jq for communicating with the Testing Farms API and parsing the responses.

API key to the Testing Farm MUST be stored in your organization's secrets to successfully access its infrastructure. See Testing Farm onboarding guide for information how to onboard to Testing Farm.

Setting update_pull_request_status input to true requires information of Pull Request's HEAD SHA value. Therefore before calling this GitHub Action, the GitHub repo must be first cloned and checked out on the Pull Request, in order to obtain the correct SHA value of Pull Request's HEAD commit. Alternatively, the HEAD SHA of the Pull Request can be provided as a pr_head_sha input.

Compatibility Notes

⚠ Currently only testing of copr builds is supported by the action. See Testing Farm docs for more information on supported test artifacts which Testing Farm can install into the environment.

Action Inputs

Testing Farm

Input Name	Description	Default value
`api_key`	Testing farm API key	empty, required from user
`api_url`	Testing farm API server url	https://api.dev.testing-farm.io/v0.1
`tf_scope`	Define the scope of Testing Farm. Possible options are 'public' or 'private'	public

Tmt Metadata

Input Name	Description	Default value
`git_url`	An url to the repository with tmt metadata	empty, required from user
`git_ref`	A tmt tests branch which will be used for tests	master
`tmt_plan_regex`	A regular expression used to select tmt plans	all
`tmt_context`	A mapping of tmt context variable tmt-context, variables separated by ;	empty

Test Environment

Input Name	Description	Default value
`compose`	Compose to run tests on. Available composes.	Fedora
`arch`	Define an architecture for testing environment	x86_64
`variables`	Environment variables for test env, separated by ;	empty
`secrets`	Environment secrets for test env, separated by ;	empty

Test Artifacts

Input Name	Description	Default value
`copr`	Copr name to use for the artifacts	epel-7-x86_64
`copr_artifacts`	`fedora-copr-build` artifacts for testing environment, separated by ;	empty

Miscellaneous

Input Name	Description	Default value
`github_token`	Github token passed from secrets	`${{ github.token }}`
`create_issue_comment`	If GitHub action will create a github issue comment	false
`pull_request_status_name`	GitHub pull request status name	Fedora
`debug`	Print debug logs when working with testing farm	true
`update_pull_request_status`	Action will update pull request status. Default: true	true
`environment_settings`	Pass custom settings to the test environment. Default: {}	empty
`pr_head_sha`	SHA of the latest commit in PR. Used for communication with GitHub API.	$(git rev-parse HEAD)

Example

Pull request example

The example below shows how the sclorg/testing-farm-as-github-action action can be used to schedule tests on Testing Farm.

name: Schedule test on Testing Farm
on:
  issue_comment:
    types:
      - created

jobs:
  tests:
    runs-on: ubuntu-20.04
    # Let's schedule tests only on user request. NOT automatically.
    # Only repository owner or member can schedule tests
    if: |
      github.event.issue.pull_request
      && contains(github.event.comment.body, '[test]')
      && contains(fromJson('["OWNER", "MEMBER"]'), github.event.comment.author_association)
    steps:
      - name: Checkout repo and switch to corresponding pull request
        uses: actions/checkout@v2
        with:
          git_ref: "refs/pull/${{ github.event.issue.number }}/head"

      - name: Schedule test on Testing Farm
        uses: sclorg/testing-farm-as-github-action@v1
        with:
          api_key: ${{ secrets.TF_API_KEY }}
          git_url: https://github.com/sclorg/sclorg-testing-farm
          tmt_plan_regex: "centos"
          pull_request_status_name: "CentOS 7"

and as soon as the job is finished you will see the test results in the pull request status:

✅ | ❌ Testing Farm - CentOS 7 - Build finished

Run workflow at push to the main branch.

The example below shows how the sclorg/testing-farm-as-github-action action can be used when pull request is merged.

name: Testing repository by Testing Farm when pull request is merged
on:
  push:
    branches:
      - main

jobs:
  tests:
    runs-on: ubuntu-20.04
    # Let's tests the repository when pull request is merged
    steps:
      - name: Schedule test on Testing Farm
        uses: sclorg/testing-farm-as-github-action@v1
        with:
          api_key: ${{ secrets.TF_API_KEY }}
          git_url: https://github.com/sclorg/sclorg-testing-farm
          tmt_plan_regex: "centos"
          update_pull_request_status: "false"

thozza / testing-farm-as-github-action