mpalmer / action-validator

Tool to validate GitHub Action and Workflow YAML files

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Ability to restrict runs-on

TWiStErRob opened this issue · comments

Intro

Hey, following recent changes, essentially any value of runs-on: is accepted. Since one of the outcomes of this action is to reduce the possibility of broken workflows, I think it would be very beneficial to validate runs-on somehow.

Proposal

  • By default validate against freely available GitHub Hosted runners (no XL, no self-hosted)
    This way the validator brings immediate value to hopefully most users,
  • With a CLI parameter (--allowed-runners) allow setting valid (incl. custom) values.
    but also gives the ability to override if needed.

Syntax

  • Comma separated list
  • Prefix @ for built-in (in the validator) groups
  • Prefix - for exclusion
  • No - prefix = positive listing

Built-in groups

(might be more):

  • github-free: ubuntu-latest, ubuntu-22.04, etc.
  • github-latest: ubuntu-latest, windows-latest, etc.
  • github-xl: -xl and -cores suffixed ones
  • self-hosted: special value for current 0.5.4 behaviour

Custom groups

If there's a known list of self hosted runners, it's possible to just list them all and use no groups. That is, to define a custom group, users need to list all runners explicitly.

For reusability the custom groups could be extracted into environment variables / organization level variables, like ACTION_VALIDATOR_ALLOWED_RUNNERS, and then that used in CLI with shell expansion: --allowed-runners=${ACTION_VALIDATOR_ALLOWED_RUNNERS},-ubuntu-latest.

Calculation

  • Start with empty set
  • Add all positive groups, and positive runners
  • Remove all negative groups and runners

i.e. order of flags doesn't matter.

Examples

  1. the default when no argument provided
--allowed-runners=!github-free
  1. a GitHub Enterprise user might validate like this, if they allow 2 self-hosted runners, and want to be explicit about versions:
--allowed-runners=!github-free,@github-large,-@github-latest,my-special-runner1,my-special-runner2
  1. an open-source community member GitHub user, who wants to use latest without automagic updates:
--allowed-runners=ubuntu-22.04,windows-2022,macos-12

Hi Robert, thanks for starting this convo.

I'm definitely in favour of introducing a way to better validate runs-on values. Not being a custom runner user myself, I'm probably not in a strong position to define a specification for how to do it. Thus, I can at best kibbitz on the work of others.

The only feedback I've got on your proposal so far is that I'm wary of using ! in a command-line argument, as shells have a habit of treating that character specially. I know of programs that use @ as a prefix for group-like behaviour; would that be an acceptable prefix in this case?

Thanks. I agree with @, changed above and reformatted a bit.

I'm not a self-hosted user either, so let's get opinions from those who do: @MPV (#59), @mathew-fleisch (#5), @deviantintegral (#51)

👏 This all looks good to me. It gives us a way in CI to make sure runners are valid, catching typos in custom runner names.