Dagger `install-only` doesn't persist `age-key`
grouville opened this issue · comments
Problem
Several users have felt the need to use dagger with the install-only
option for 2 reasons mainly:
- They need to retrieve the context of the commit / PR
- Their internal policy restricts the use of encrypted files containing secrets, regardless the security level of the encrypted file.
For the second case, no issue arises
People just push without a .dagger
folder and literally add the inputs/secrets in the GitHub workflow:
# This is a basic workflow to help you get started with Actions
name: Dagger PR test
# Controls when the workflow will run
on:
# Triggers the workflow on a pull request event
pull_request:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "dagger"
dagger:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Install Dagger
uses: dagger/dagger-action@v1
with:
install-only: true
#- name: Show Dagger version
#run: dagger version
- name: Setup dagger environment "myenv"
run: |
dagger init
dagger new myenv -p ./myenv
- name: Pass PR number to Dagger
run: dagger input text prID ${{github.event.number}} -e myenv
- name: List Dagger inputs
run: dagger input list -e myenv
- name: Dagger up!
run: dagger up -e myenv
- name: List Dagger outputs
run: dagger output list -e myenv
But, some people want to retrieve part of the Github context AND store their secrets in the sops file:
jobs:
test_ci:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Install Dagger
uses: dagger/dagger-action@v1
with:
age-key: ${{ secrets.DAGGER_AGE_KEY }}
install-only: true
- name: Pass Commit SHA to Dagger
run: dagger input text githubCommitSha ${{github.sha}}
- name: Dagger up!
run: dagger up
The above code doesn't work, dagger doesn't find the sops key
Current Solution
After some debug, I found a way to make it work. It seems a little weird because the second time we added an input, the sops key gets found.
name: Dagger PR test
# Controls when the workflow will run
on:
# Triggers the workflow on a pull request event
push:
pull_request:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "dagger"
dagger:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Install Dagger
uses: dagger/dagger-action@v1
with:
install-only: true
- name: Pass commit sha
uses: dagger/dagger-action@v1
with:
age-key: ${{ secrets.DAGGER_AGE_KEY }}
args: input text githubCommitSha ${{github.sha}}
- name: Dagger new input
run: dagger input text githubCommitNumber "ok"
- name: Dagger up!
run: dagger up
Btw, this will become obsolete with dagger europa
, as env var will be retrievable from the context
As discussed with @helderco (thanks for your point of view), this creates a weird DX issue.
The doc needs to be updated + a PR extending it needs to be proposed (doing it right now)
Can you put it in its own step?
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Install Dagger
uses: dagger/dagger-action@v1
with:
install-only: true
- name: Set up age key
uses: dagger/dagger-action@v1
with:
age-key: ${{ secrets.DAGGER_AGE_KEY }}
- name: Pass commit sha
run: dagger input text githubCommitSha ${{github.sha}}
- name: Dagger new input
run: dagger input text githubCommitNumber "ok"
- name: Dagger up!
run: dagger up
That's an excellent idea. But this doesn't work:
Run dagger/dagger-action@v1
[8](https://github.com/grouville/test-ci/runs/5277022914?check_suite_focus=true#step:4:8)
Downloading https://dl.dagger.io/dagger/releases/0.2.0-alpha.5/dagger_v0.2.0-alpha.5_linux_amd64.tar.gz
[9](https://github.com/grouville/test-ci/runs/5277022914?check_suite_focus=true#step:4:9)
Extracting Dagger
[10](https://github.com/grouville/test-ci/runs/5277022914?check_suite_focus=true#step:4:10)
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/ede94a77-8382-4663-9396-2effec74b1df -f /home/runner/work/_temp/cfda0e10-c31e-4040-a6f7-a5fc3fae864d
[11](https://github.com/grouville/test-ci/runs/5277022914?check_suite_focus=true#step:4:11)
Error: args input required
age-key
needs an arg
to work. Making an exception like that is possible though
- name: Install Dagger
uses: dagger/dagger-action@v1
with:
age-key: ${{ secrets.DAGGER_AGE_KEY }}
install-only: true
Ah, looking at the code it makes sense. Because everything else is trying to run a dagger command.
Yes it makes sense to me having the age-key being included as part of the installation process.
Changing the order still fails, as the install-only returns and is the only option for which an arg is not required.
With the changes I was thinking about:
- install only possible
- install only + age-key possible
- any command + arg possible
I think we could mutualize the steps using an input list like:
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
- name: Dagger
uses: dagger/dagger-action@v1
with:
age-key: ${{ secrets.DAGGER_AGE_KEY }}
cmds: |
input text githubCommitSha ${{github.sha}}
input text githubCommitNumber "ok"
up
So it would be self-contained in the same action. WDYT?
I think we could mutualize the steps using an input list like:
# Steps represent a sequence of tasks that will be executed as part of the job steps: # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it - uses: actions/checkout@v2 - name: Dagger uses: dagger/dagger-action@v1 with: age-key: ${{ secrets.DAGGER_AGE_KEY }} cmds: | input text githubCommitSha ${{github.sha}} input text githubCommitNumber "ok" upSo it would be self-contained in the same action. WDYT?
That feels smooth, you're right. And I can't seem to see a use-case where this would bring a DX issue.
One (probably dumb) question
As we default the version to latest
when the version
step is not specified. Wouldn't that break the current users configs or is it scoped to some versions ? In other words, wouldn't that bring retrocompatibility issues ?
Another question
Does that mean that we shall remove the install-only
option ?
What about the workdir
option ? As dagger has the --project
CLI command, is it still relevant ?
Europa will also remove that necessity I believe
From your proposal, the new steps
would be:
Following inputs can be used as step.with
keys
Name | Type | Default | Description |
---|---|---|---|
version | String | latest | Dagger version |
age-key | String | Dagger private key | |
workdir | String | . | Working directory (below repository root) |
cmds | List<String> | List of Dagger commands to execute | |
cleanup | Bool | true | Cleanup Dagger home folder at the end of a job |
WDYT @shykes ?
I don't have a strong opinion since this is specific to 0.1 and we are releasing 0.2 in the coming weeks.
I am more concerned about how:
- How to cleanly migrate users of this action from Dagger 0.1 to 0.2
- How the migration interacts with #21
How to cleanly migrate users of this action from Dagger 0.1 to 0.2
As GHA best practice, and as pointed out in Dagger docs, users have been (at least should have been) pinning to a specific version of this action rather than main
. Excerpt from GHA tutorial on our docs site: uses: dagger/dagger-action@v1
We can safely publish a 0.2-based action as uses: dagger/dagger-action@v2
without breaking any 0.1 user.
Note that:
v1
/v2
: that's the version of the GH action, reserved for breaking changes in the action definition itself (e.g. we might change the syntax because ofdagger do
)- Additionally, there's a
version
field in the action which defines which version of dagger to use (defaults tolatest
)
We can:
- Force
v1
to use the latest0.1.x
version (rather than latest) - Have a
v2
with the latest syntax that points tolatest
(0.2.x
)
How the migration interacts with #21
AFAIK (@crazy-max to confirm), it should run smoothly. GH sets up a "redirect" after renaming a repo. Even though the new canonical name will be different, the old name can still be used interchangeably.
Not relevant anymore with new version