reviewdog erroneously falls back to logging on forked repo even when it has write permissions

[James Pearson Hughes]

I'm debugging Github Actions jobs where the token has write permissions:

but reviewdog says it does not, and only logs messages:

I've traced this down in code. The github-pr-review reporter checks to see if the token is read-only. HasReadOnlyPermissionGitHubToken however only checks to see if it's in a fork. However, github now provides a way to send write tokens to forks:

You can use the permissions key to add and remove read permissions for forked repositories, but typically you can't grant write access. The exception to this behavior is where an admin user has selected the Send write tokens to workflows from pull requests option in the GitHub Actions settings.

So, we can't assume that all forks won't have write access; it needs to actually check the access of the token, or alternatively, try to post comments and if that fails then do the fallback action.

[James Pearson Hughes]

As a second thing, I don't feel like this should be silently falling back to logs. If you had a working setup and then it stopped working (someone changed the GitHub token permissions, a bug in reviewdog gets pulled in, GitHub changes something), you'd be unlikely to notice that it broke until someone spotted a linting issue that should've been caught by a linter and investigated why it wasn't. If the reporter is set to "post comments on prs", then failure to post comments on prs should produce a failure status on the Action.

That might be an architectural problem with the reviewdog runners though, now that I think about it. I guess you'd need to have certain exit codes for "we found issues" and other ones for "reviewdog itself had errors" and do different things in the callers depending on that... hmm.

[James Pearson Hughes]

@haya14busa I decided to take a look at trying to fix this myself. My conclusion is that we should probably just revert 2238fbd. Graceful degredation is nice in theory, but in practice it means that the error is hidden. I had a pr that should have triggered comments via action-shellcheck, but there were no comments and everything was green:

It's only when clicking into the Actions run and digging into the annotations that there's any indication that it did find linter issues. If something is wrong with the Github tokens, no one is ever going to know unless the job starts failing.

(And also there's the initial root issue reported here, which is that the check is incorrect and there actually is permission in this case. But if we get rid of the check then that bypasses the bug.)