Export to GHA cache fails with an error
tlinhart opened this issue · comments
What happened?
I'm trying to build a Docker image using the new Docker Build provider while employing the GitHub Actions cache as per the documentation. The Pulumi program runs in a GHA workflow job as outlined below. However, exporting the cache fails with an error.
Example
This is the relevant part of the Pulumi program:
image_name = pulumi.Output.concat(repository.repository_url, ":latest")
if os.getenv("GITHUB_ACTIONS"):
cache_from = docker_build.CacheFromArgs(
gha=docker_build.CacheFromGitHubActionsArgs()
)
cache_to = docker_build.CacheToArgs(
gha=docker_build.CacheToGitHubActionsArgs(
mode=docker_build.CacheMode.MAX, ignore_error=True
)
)
else:
cache_from = docker_build.CacheFromArgs(
registry=docker_build.CacheFromRegistryArgs(ref=image_name)
)
cache_to = docker_build.CacheToArgs(
inline=docker_build.CacheToInlineArgs()
)
image = docker_build.Image(
"xxx",
context=docker_build.BuildContextArgs(location=".."),
platforms=[docker_build.Platform.LINUX_AMD64],
tags=[image_name],
cache_from=[cache_from],
cache_to=[cache_to],
push=True,
registries=[
docker_build.RegistryArgs(
address=repository.repository_url,
username=auth_token.user_name,
password=pulumi.Output.secret(auth_token.password),
)
],
)
This is the relevant part of the GitHub Actions workflow:
name: CI
on:
push:
branches:
- "**"
permissions:
contents: read
id-token: write
jobs:
infra:
name: Provision infrastructure
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: eu-central-1
- name: Create or update stack resources
uses: pulumi/actions@v5
with:
command: up
refresh: true
diff: true
suppress-progress: true
stack-name: xxx-prod
work-dir: infra
cloud-url: s3://xxx
env:
PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
This is the relevant part of the GitHub Actions log from the pulumi up step:
#51 [auth] sharing credentials for xxx.dkr.ecr.eu-central-1.amazonaws.com
#51 DONE 0.0s
#50 exporting to image
#50 pushing layers 6.9s done
#50 pushing manifest for xxx.dkr.ecr.eu-central-1.amazonaws.com/xxx-ac3cfb0:latest@sha256:xxx
#50 pushing manifest for xxx.dkr.ecr.eu-central-1.amazonaws.com/xxx-ac3cfb0:latest@sha256:xxx 0.6s done
#50 DONE 17.0s
#52 exporting to GitHub Actions Cache
#52 preparing build cache for export
#52 preparing build cache for export 19.5s done
#52 ERROR: failed to parse error response 404: : unexpected end of JSON input
Output of pulumi about
CLI
Version 3.116.1
Go Version go1.22.2
Go Compiler gc
Host
OS ubuntu
Version 24.04
Arch x86_64
Backend
Name xxx
URL s3://xxx
User xxx
Organizations
Token type personal
Pulumi locates its logs in /tmp by default
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
@tlinhart sorry, the error message from Docker is really unhelpful here. I suspect you simply need to expose the appropriate credentials to your workflow. The docs recommend this one.
An action like
crazy-max/ghaction-github-runtimeis recommended to expose appropriate credentials to your GitHub workflow.
@blampe yeah I know but I don't know how to provide a better one as Pulumi's GHA action doesn't seem to have any option for setting verbosity or debugging output.
The error doesn't seem to be related to credentials. I tested using the crazy-max/ghaction-github-runtime action and the output stayed the same. When I explicitely set the ACTIONS_RUNTIME_TOKEN variable to a dummy value, it told me that I lack permissions so I guess the variables are read from environment correctly. Also they should be as they are exposed to the actions, they are only invisible to the inline scripts (i.e. run steps). I found it on multiple places but it's mentioned at least here and here.
@tlinhart I did notice our docs referred to ACTIONS_RUNTIME_URL when the correct name is actually ACTIONS_CACHE_URL, if that matters. As far as I can tell that shouldn't affect the behavior but I'll try to confirm that.
With the ghaction-github-runtime action, make sure you're using v3 and the step happens before your Pulumi update.
@blampe yeah that's exactly how I was trying to use that, the step was placed between aws-actions/configure-aws-credentials@v4 and pulumi/actions@v5 in the workflow I posted in the issue.
Reading your comment about using a wrong env variable should actually be the source of the problem as I think the error message refers to GitHub API returning a 404 response.
@blampe yep, I can confirm the wrong env variable was the issue. I tried to fix the problem with
os.environ["ACTIONS_RUNTIME_URL"] = os.environ["ACTIONS_CACHE_URL"]
in the Pulumi program and it started to work – no error in the GHA workflow log and I can see the artifacts in the GitHub Actions Caches.
@tlinhart interesting, thanks for confirming that. I was under the impression this would still get picked up in a lower layer (ref) but I see now it must be the case that it must be set explicitly.
#80 should take care of this for you. I might try to include a couple other fixes before cutting a new release.
Cool, thanks a lot!
When it comes to the former, I read it this way. When the url argument in GHA's cache_from and cache_to is not set explicitely, you prime it with the value of incorrect env variable ACTIONS_RUNTIME_URL (see here and here). These values (i.e. URL, token etc.) are then explicitely provided to the underlying Docker build command which suppresses using the default value from the correct env variable (see here). The Docker build command is what actually uses the go-actions-cache library, not Pulumi provider. But that's just my interpretation of what happens, might be a different story :-)