Run development version of dagger?
shykes opened this issue · comments
My plan requires a patched version of dagger. I want to run it in a github workflow, but since it runs the latest official release by default (currently 0.2.0), my patch is not included. How can I configure dagger-for-github to run my patched version of dagger?
Example that doesn't use dagger-for-github:
https://gist.github.com/maert/47cc6b3a30ed3f0efc650ac88ba47e03
If the interface is good, could be made into action inputs with the dagger executable put in an appropriate spot in PATH.
This is what I ended up doing for the todoapp, before 0.2.0 was released: https://github.com/dagger/dagger/blob/c3f21958d2d64ca6da2704b6b1f644d46802def0/.github/workflows/todoapp.yml
I definitely think that we should support this in the action.
Example that doesn't use dagger-for-github: https://gist.github.com/maert/47cc6b3a30ed3f0efc650ac88ba47e03
If the interface is good, could be made into action inputs with the dagger executable put in an appropriate spot in PATH.
Thanks, I'm trying to run this but the action fails with "Error: Input required and not supplied: token"
- How do I provide the missing input?
- Is there a way to require no input, for example by using the credentials already available to the GHA runner by default?
Disclaimer: I am a GHA noob.
Here is my workflow, in case it's helpful:
on:
push:
branches: [main]
pull_request:
branches: [main]
env:
DAGGER_LOG_FORMAT: plain
jobs:
dagger:
runs-on: ubuntu-latest
steps:
- name: Checkout dagger at ref
uses: actions/checkout@v3
with:
repository: ${{ github.event.inputs.dagger-repo }}
path: 'src/dagger'
ref: main
token: ${{ secrets.GH_PAT }}
fetch-depth: 0
# get SHA hash in case branch or tag was used for checkout
- name: Get dagger git ref hash
id: dagger-git-ref
run: |
cd ${{ github.workspace }}/src/dagger
echo "::set-output name=hash::$(git rev-parse HEAD)"
- name: Install go
uses: actions/setup-go@v2
with:
go-version: 1.16
- name: Cache dagger binary
id: cache-dagger-binary
uses: actions/cache@v2
with:
path: ${{ github.workspace }}/src/dagger/cmd/dagger/dagger
key: ${{ runner.os }}-cache-dagger-binary-${{ steps.dagger-git-ref.outputs.hash }}
- name: Build dagger
if: steps.cache-dagger-binary.outputs.cache-hit != 'true'
run: |
cd ${{ github.workspace }}/src/dagger
make dagger
- name: Run dagger version
run: |
${{ github.workspace }}/src/dagger/cmd/dagger/dagger version
##
## ACTUAL ACTION WORKLOAD BELOW:
##
- name: Clone repository
uses: actions/checkout@v2
- name: Generate asciidoc
# https://github.com/dagger/dagger-for-github
run: |
${{ github.workspace }}/src/dagger/cmd/dagger/dagger do -p ./objectives render
Example that doesn't use dagger-for-github: https://gist.github.com/maert/47cc6b3a30ed3f0efc650ac88ba47e03
If the interface is good, could be made into action inputs with the dagger executable put in an appropriate spot in PATH.Thanks, I'm trying to run this but the action fails with
"Error: Input required and not supplied: token"
- How do I provide the missing input?
- Is there a way to require no input, for example by using the credentials already available to the GHA runner by default?
Update: I removed the line setting an explicit token (token: ${{ secrets.GH_PAT }}
) in the hope that it would use a default token. It appears to have worked, now I fail a bit later at "build dagger":
Run cd /home/runner/work/shykes/shykes/src/dagger
[8](https://github.com/dagger/shykes/runs/5579280314?check_suite_focus=true#step:6:8)
make: *** No rule to make target 'dagger'. Stop.
[9](https://github.com/dagger/shykes/runs/5579280314?check_suite_focus=true#step:6:9)
Error: Process completed with exit code 2.
I thought maybe make
is being run in the wrong directory, but I don't know how to troubleshoot this.
How do I provide the missing input?
In the example above, notice secrets.GH_PAT
. You need to create a secret called GH_PAT
.
I refined things and made an actual resuable lego brick, an action, here:
https://github.com/maert/build-dagger-github-action/blob/main/action.yml#L4-L15
In this one, the inputs are similar, but I make the repo-token
optional and give good defaults for the other two required inputs.
You can use the action like this to install dagger
from dagger/dagger
and main
branch:
name: Build dagger workflow
on: [push]
jobs:
build-dagger:
runs-on: ubuntu-latest
steps:
- uses: maert/build-dagger-github-action@v1.3
A more complete example that performs a checkout out of the repo (with a dagger cue file in it), builds dagger from source at a particular git commit in my fork, and then runs a dagger do
here: https://github.com/maert/test-action/blob/main/.github/workflows/test.yml
Nice job @maert! 🙌🏻
I am keen on folding it into this repository. I expect this to remain in my swap for a few weeks, but I intend to come back to it.
Point-in-time reference: https://github.com/gerhard/shykes/commit/d1bbe356ea425557418f31e8e7aadf978e8dffcc (private repo, open to @shykes as it's a fork)
okr.yaml
that works https://github.com/gerhard/shykes/runs/5589675700?check_suite_focus=true#step:7:65 (private repo, open to @shykes as it's a fork)
name: okr
on:
push:
branches:
pull_request:
branches:
jobs:
generate:
runs-on: ubuntu-latest
steps:
- name: Clone this repository
uses: actions/checkout@v3
- name: Clone dagger repository
uses: actions/checkout@v3
with:
repository: dagger/dagger
# https://github.com/actions/checkout#Checkout-multiple-repos-private
token: ${{ secrets.DAGGER_SHYKES_OKR_GH_PAT }}
path: 'src/dagger'
ref: main
fetch-depth: 0
- name: Install go
uses: actions/setup-go@v2
with:
go-version: 1.18
- name: Build dagger
run: |
cd ${{ github.workspace }}/src/dagger
make dagger
- name: Verify dagger version
run: |
${{ github.workspace }}/src/dagger/cmd/dagger/dagger version
- name: Generate asciidoc
env:
DAGGER_LOG_FORMAT: plain
run: |
${{ github.workspace }}/src/dagger/cmd/dagger/dagger do -p ./objectives render
Thank you both for your help! I got it to work on my repo.
My 2c on how to implement this feature:
- We already have a
version
field - The
version
field could be extended in the following way: if its value is a URL, thedagger
binary is downloaded from that URL instead of the default release URL and added to the PATH.
This has the advantage of limiting proliferation of custom builds of dagger, which is a subtle form of DX fragmentation. For example if we change our build tool from make
to dagger
, we don't want to deal with outdated GHA configurations that still run make
.
Short term, each user can build their own binary and publish it anywhere: github, s3 etc.
Longer term, we can (and IMO should) host a binary-on-demand service eg. get.dagger.io/binary/dagger-$GITREF.tgz
that builds the requested version just-in-time for download.
We have something like this with our setup-buildx-action
. You can define a git repo as version
and it will build on-fly buildx and use it: https://github.com/docker/setup-buildx-action/runs/5987865407?check_suite_focus=true#step:3:95
name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
with:
version: https://github.com/docker/buildx.git#refs/pull/731/head
It's quite powerful when you want to test a PR with a specific context. Under the hood it uses a Git remote context to build buildx with buildx 😅:
docker buildx build --target binaries --output type=local,dest=/tmp/docker-setup-buildx-8FFA7Y/out https://github.com/docker/buildx.git#refs/tags/v0.5.1
Doing something like this should be possible for dagger too. Here is the PR for ref: docker/setup-buildx-action#99
We have something like this with our
setup-buildx-action
. You can define a git repo asversion
and it will build on-fly buildx and use it: https://github.com/docker/setup-buildx-action/runs/5987865407?check_suite_focus=true#step:3:95Doing something like this should be possible for dagger too. Here is the PR for ref: docker/setup-buildx-action#99
This is great! Thank you!
#53 should fix this issue but depends on dagger/dagger#2288 if we want to build from the default branch.
In the meantime you can try with:
jobs:
deploy:
runs-on: ubuntu-latest
steps:
-
name: Checkout
uses: actions/checkout@v3
-
name: Dagger
uses: crazy-max/dagger-action@build-ref
with:
version: https://github.com/dagger/dagger.git#refs/pull/2288/head
cmds: do test
@crazy-max You save my life!
I have mirgated to dagger with my PR now dagger/dagger#2310 (https://github.com/octohelm/cuemod)