`project_id` doesn't appear to set default project correctly

Question

`project_id` doesn't appear to set default project correctly

nickstenning opened this issue 2 years ago · comments

TL;DR

Specifying a project_id doesn't appear to set the default project correctly, at least when using Workload Identity Federation, preferring instead the home project of the service account.

Expected behavior

    - name: 'Authenticate to Google Cloud'
      uses: 'google-github-actions/auth@v1'
      with:
        workload_identity_provider: 'projects/123456/locations/global/workloadIdentityPools/github/providers/github-actions'
        service_account: 'deploy@giraffe-project.iam.gserviceaccount.com'

    - name: 'Set up Cloud SDK'
      uses: 'google-github-actions/setup-gcloud@v1'
      with:
        project_id: rhino-project

    - name: 'Fetch cluster credentials'
      run: gcloud container clusters get-credentials my-cluster --region us-central1

I expect the gcloud container ... command to run with a default project of rhino-project.

Observed behavior

Instead it runs with the default project of giraffe-project, as can be seen from the logs:

Authenticate to Google Cloud

Run google-github-actions/auth@v1
  with:
    workload_identity_provider: projects/123456/locations/global/workloadIdentityPools/github/providers/github-actions
    service_account: deploy@giraffe-project.iam.gserviceaccount.com
    create_credentials_file: true
    export_environment_variables: true
    cleanup_credentials: true
    access_token_lifetime: 3600s
    access_token_scopes: https://www.googleapis.com/auth/cloud-platform
    retries: 0
    id_token_include_email: false
  env:
  
Created credentials file at "/home/runner/work/api/api/gha-creds-3d2bfbb1ad9b4.json"

Set up Cloud SDK

Run google-github-actions/setup-gcloud@v1
  with:
    project_id: rhino-project
    version: latest
  env:
    CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE: /home/runner/work/api/api/gha-creds-43dbf6bb1ad98b4.json
    GOOGLE_APPLICATION_CREDENTIALS: /home/runner/work/api/api/gha-creds-4d2bf6bb1ad98b.json
    GOOGLE_GHA_CREDS_PATH: /home/runner/work/api/api/gha-creds-43d2bf6bb1ad98b4.json
    CLOUDSDK_CORE_PROJECT: giraffe-project
    CLOUDSDK_PROJECT: giraffe-project
    GCLOUD_PROJECT: giraffe-project
    GCP_PROJECT: giraffe-project
    GOOGLE_CLOUD_PROJECT: giraffe-project
  
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/2793fbb-3920-452f-9244-a765d3b51c -f /home/runner/work/_temp/dde1c12b-6302-4245-a4-3f01e5adf82f
Successfully authenticated
Successfully set default project

Fetch cluster credentials

Run gcloud container clusters get-credentials my-cluster --region us-central1
  gcloud container clusters get-credentials my-cluster --region us-central1
  shell: /usr/bin/bash -e {0}
  env:
    CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE: /home/runner/work/api/api/gha-creds-43dbf6bb1ad98b4.json
    GOOGLE_APPLICATION_CREDENTIALS: /home/runner/work/api/api/gha-creds-4d2bf6bb1ad98b.json
    GOOGLE_GHA_CREDS_PATH: /home/runner/work/api/api/gha-creds-43d2bfbb1ad9b4.json
    CLOUDSDK_CORE_PROJECT: giraffe-project
    CLOUDSDK_PROJECT: giraffe-project
    GCLOUD_PROJECT: giraffe-project
    GCP_PROJECT: giraffe-project
    GOOGLE_CLOUD_PROJECT: giraffe-project
    CLOUDSDK_METRICS_ENVIRONMENT: github-actions-setup-gcloud
  
Fetching cluster endpoint and auth data.
ERROR: (gcloud.container.clusters.get-credentials) ResponseError: code=403, message=Required "container.clusters.get" permission(s) for "projects/giraffe-project/locations/us-central1/clusters/my-cluster".
Error: Process completed with exit code 1.

Action YAML

name: CI

on:
  push:
    branches:
      - main
  pull_request:

# Ensure only one deployment runs at a time
concurrency:
  group: ${{ github.ref }}
  cancel-in-progress: true

env:
  GKE_CLUSTER: 'cog-cluster-us-central1'
  # Cluster is currently in a different project from the deploy service account
  GKE_CLUSTER_PROJECT: 'replicate'
  GKE_REGION: 'us-central1'

jobs:
  test:
    name: "Test"
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-go@v3
      with:
        go-version-file: go.mod
        cache: true
    - run: "make test"
      name: Run test

  lint:
    name: "Lint"
    if: ${{ github.event_name == 'pull_request' }}
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-go@v3
      with:
        go-version-file: go.mod
        cache: false # for this job we rely on golangci-lint-action's cache
    - name: Lint
      uses: golangci/golangci-lint-action@v3
      with:
        version: v1.50
    - run: "make check-fmt"
      name: Formatting

  build:
    name: "Build image"
    runs-on: ubuntu-latest
    if: ${{ github.actor != 'dependabot[bot]' && !github.event.pull_request.head.repo.fork }}
    needs: test
    outputs:
      image: ${{ steps.build.outputs.image }}
    steps:
      - uses: actions/checkout@v3
      - uses: 'google-github-actions/auth@v0'
        with:
          credentials_json: '${{ secrets.BUILDER_JSON_KEY }}'
          service_account: 'builder@replicate.iam.gserviceaccount.com'
      - uses: 'google-github-actions/setup-gcloud@v0'
      - name: 'Submit build'
        id: 'build'
        run: './scripts/build.sh'

  end-to-end-test:
    name: "End-to-end test"
    if: ${{ github.event_name == 'pull_request' && github.actor != 'dependabot[bot]' && !github.event.pull_request.head.repo.fork }}
    runs-on: ubuntu-latest
    needs: build

    # Inject token for Google Workload Identity Federation auth
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-go@v3
      with:
        go-version-file: go.mod
        cache: true

    - name: 'Authenticate to Google Cloud'
      uses: 'google-github-actions/auth@v1'
      with:
        # This uses workload identity federation, and requires that the
        # appropriate policy binding has been set up for this repository. See if
        # it has with:
        #
        #     gcloud iam service-accounts get-iam-policy deploy@replicate-production.iam.gserviceaccount.com
        #
        workload_identity_provider: 'projects/1025538909507/locations/global/workloadIdentityPools/github/providers/github-actions'
        service_account: 'deploy@replicate-production.iam.gserviceaccount.com'

    - name: 'Set up Cloud SDK'
      uses: 'google-github-actions/setup-gcloud@v1'
      with:
        project_id: "${{ env.GKE_CLUSTER_PROJECT }}"

    - name: 'Fetch cluster credentials'
      run: gcloud container clusters get-credentials "$GKE_CLUSTER" --region "$GKE_REGION"

    - name: "Run test"
      run: "make test-end-to-end"
      env:
        DOCKER_IMAGE: "${{ needs.build.outputs.image }}"
        # TODO: this is using andreas' token in CI.
        # we should make a dev token.
        REPLICATE_API_TOKEN: "${{ secrets.REPLICATE_API_TOKEN }}"

  deploy:
    if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
    name: "Deploy"
    runs-on: ubuntu-latest
    needs: build

    # Inject token for Google Workload Identity Federation auth
    permissions:
      contents: 'read'
      id-token: 'write'

    steps:
      - uses: actions/checkout@v3

      - name: 'Authenticate to Google Cloud'
        uses: 'google-github-actions/auth@v1'
        with:
          # This uses workload identity federation, and requires that the
          # appropriate policy binding has been set up for this repository. See if
          # it has with:
          #
          #     gcloud iam service-accounts get-iam-policy deploy@replicate-production.iam.gserviceaccount.com
          #
          workload_identity_provider: 'projects/1025538909507/locations/global/workloadIdentityPools/github/providers/github-actions'
          service_account: 'deploy@replicate-production.iam.gserviceaccount.com'

      - name: 'Set up Cloud SDK'
        uses: 'google-github-actions/setup-gcloud@v1'
        with:
          project_id: "${{ env.GKE_CLUSTER_PROJECT }}"

      - name: 'Fetch cluster credentials'
        run: gcloud container clusters get-credentials "$GKE_CLUSTER" --region "$GKE_REGION"

      - name: Deploy
        run: './scripts/deploy.sh ${{ needs.build.outputs.image }}'

      - name: Add deploy marker to Honeycomb
        run: |
          first_line=$(echo "${{ github.event.head_commit.message }}" | tr '\n' '\f' | sed -e 's/^\([^\f]*\).*$/\1/')
          curl https://api.honeycomb.io/1/markers/api -X POST -H "X-Honeycomb-Team: ${{ secrets.HONEYCOMB_API_KEY }}" -d "{\"message\": \"$first_line\", \"url\": \"${{ github.event.head_commit.url }}\", \"type\": \"deploy\"}"

Log output

See "Observed behavior" section.

Additional information

No response

Seth Vargo · Answer 1 · Fri Dec 16 2022 22:44:47 GMT+0800 (China Standard Time)

I think gcloud is preferring the environment variables over the core value. Can you set project_id in the auth section instead?

- uses: 'google-github-actions/auth@v1'
  with:
    workload_identity_provider: 'projects/123456/locations/global/workloadIdentityPools/github/providers/github-actions'
    service_account: 'deploy@giraffe-project.iam.gserviceaccount.com'
    project_id: 'rhino-project'

Also, your workflows will be must faster if you use get-gke-credentials, since it doesn't have a dependency on installing and configuring gcloud:

- uses: 'google-github-actions/auth@v1'
  with:
    workload_identity_provider: 'projects/123456/locations/global/workloadIdentityPools/github/providers/github-actions'
    service_account: 'deploy@giraffe-project.iam.gserviceaccount.com'
    project_id: 'rhino-project'

- uses: 'google-github-actions/get-gke-credentials@v1'
  with:
    cluster_name: 'my-cluster'
    location: 'us-central1-a'

Nick Stenning · Answer 2 · Fri Dec 16 2022 22:56:09 GMT+0800 (China Standard Time)

I think gcloud is preferring the environment variables over the core value. Can you set project_id in the auth section instead?

Ah, ok, I'll try this.

Also, your workflows will be must faster if you use get-gke-credentials, since it doesn't have a dependency on installing and configuring gcloud.

Thank you!

Nick Stenning · Answer 3 · Sat Dec 17 2022 17:23:48 GMT+0800 (China Standard Time)

I can confirm both that setting project_id on the auth step gets gcloud to use the correct project, and that the get-gke-credentials action is a better way of doing this in any event :)

Thanks for the help!