HttpError: Not Found (Warning: Not able to edit comment, defaulting to creating a new comment)

[Miroslav Shakhmaykin]

What happened?

I use Pulumi with self-stored backend (AWS S3), no Pulumi Cloud account created.
I only configured Preview action for now:

name: Pulumi
on:
  # Allow to manually run workflow
  workflow_dispatch:

  pull_request:
      branches:
        - 'develop'
jobs:
  preview:
    strategy:
      matrix:
        stack: [eks-cluster-go]
    name: Pulumi Preview
    runs-on: ubuntu-latest
    env:
      STACK_DIR: pulumi/aws/${{ matrix.stack }}
      PULUMI_BACKEND_URL: 's3://${{ vars.PULUMI_STATE_BUCKET }}/${{ matrix.stack }}?region=us-east-1&awssdk=v2' # workaround per https://github.com/pulumi/actions/issues/1010
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 1
      - uses: actions/setup-go@v4
        with:
          go-version: 'stable'
      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-region: ${{ secrets.AWS_REGION }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      - run: go mod download
        working-directory: ${{ env.STACK_DIR }}
      - uses: pulumi/actions@v4
        env:
          PULUMI_CONFIG_PASSPHRASE: ${{ secrets.PULUMI_CONFIG_PASSPHRASE }}
        with:
          command: preview
          cloud-url: s3://${{ vars.PULUMI_STATE_BUCKET }}/${{ matrix.stack }}
          stack-name: dev
          comment-on-pr: true
          github-token: ${{ secrets.PULUMI_GITHUB_TOKEN_ALL }}
          work-dir: ${{ env.STACK_DIR }}

Note: After initial failures with default token, I created a fine-grained Github token from inside my personal account profile and explicitly granted ALL POSSIBLE Read/Write permissions to it to test.

The job passes the Preview step well, but complains about not being able to edit comments, and throws that HttpError. It results in the failed check, and PR won't be merged.

Example

....
##[debug]Evaluating: secrets.PULUMI_GITHUB_TOKEN_ALL
##[debug]Evaluating Index:
##[debug]..Evaluating secrets:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'PULUMI_GITHUB_TOKEN_ALL'
##[debug]=> '***'
##[debug]Result: '***'
....

::group::pulumi preview on dev
>pulumi preview on dev                                ###   <--------------------------------    everything passes well here


Warning: Not able to edit comment, defaulting to creating a new comment.
HttpError: Not Found
    at /home/runner/work/_actions/pulumi/actions/v4/webpack:/pulumi-github-action/node_modules/@octokit/request/dist-node/index.js:86:1
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
##[debug]Node Action run completed with exit code 1

Output of pulumi about

warning: using pulumi-language-go from $PATH at /opt/homebrew/bin/pulumi-language-go
warning: using pulumi-language-go from $PATH at /opt/homebrew/bin/pulumi-language-go
CLI          
Version      3.87.1-dev.0
Go Version   go1.21.2
Go Compiler  gc

Plugins
NAME        VERSION
aws         5.4.0
awsx        1.0.0
docker      3.2.0
eks         0.41.0
go          unknown
kubernetes  3.17.0

Host     
OS       darwin
Version  14.0
Arch     arm64

This project is written in go: executable='/opt/homebrew/bin/go' version='go version go1.21.2 darwin/arm64'

Backend        
Name           <laptop-name-here>
URL            s3://<state-bucket-name-here>/eks-cluster-go?region=us-east-1&awssdk=v2
User           miroslav
Organizations  
Token type     personal

Dependencies:
NAME                               VERSION
github.com/pulumi/pulumi-awsx/sdk  1.0.0
github.com/pulumi/pulumi-eks/sdk   0.41.0
github.com/pulumi/pulumi/sdk/v3    3.74.0

Pulumi locates its logs in /var/folders/qn/56sv8gqj0195tt2bssfj9nk40000gn/T/ by default
warning: Failed to get information about the current stack: No current snapshot

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[Justin Van Patten]

This looks like it could be related to #838

See #838 (comment). Does changing your Workflow Permissions to allow read and write help?

[Miroslav Shakhmaykin]

Oh, this helped, thank you!

I added permissions parameter to my workflow and enabled writes to Pull requests.

Interestingly, that this with: github-token: ${{ secrets.PULUMI_GITHUB_TOKEN_ALL }} didn't help even when GITHUB_TOKEN_ALL was configured in repo secrets with write permissions.

[Justin Van Patten]

Glad that helped. I'm going to close this as a duplicate of #838