renovatebot / renovate-approve-bot

If the user has configured to dismiss PR reviews when they're stale, then Renovate Approve's approval will be removed every time the branch is rebased/force pushed. Renovate Approve needs extra logic to detect this and add a new approval.

This is among others useful when renovate opens multiple PRs in a given repo so after merging one of them, in the second one there's a force-push which dismisses approval.

Right now the logic is too simple:

renovate-approve-bot/index.js

Line 3 in e61af3f

robot.on('pull_request.opened', async context => {

But I'm not sure we can also just post approvals non-stop every time there is an edit to the PR, because IIRC they all count as separate approvals and eventually GitHub rejects it.

Is there a plan of how we might address this? There are currently a few scenarios where this comes up, including when there’s a newer version released before the merge happens.

So should the Approve bot just simply approve every time it notices Renovate Bot committing to an existing PR?

That was my first thought, although you mentioned github eventually rejects it, I’m not sure what the limit is and what the consequences are, perhaps you can enlighten us? If it’s just that eventually it stops being able to approve that particular PR then this brute force approach would still be much more resilient than the current try-once approach.

It would be great if the approve bot re-approved after each commit or force-push.

Even better it would be if it could obtain a webhook from GitHub so the PR would be re-approved within one minute instead of one hour.

Should we just listen to edited as well as opened?

Perhaps, but we need to confirm:

if the branch is force pushed to while the PR content remains the same, does the “edited” still get triggered?
if the PR is edited but the existing review is still valid, can we check for that first and avoid doing a duplicate approval?

Hey @rarkins, did you happen to make progress on this? We auto-merge @types/... dependency updates, which often come in batches, so only the first one gets merged. We also dismiss reviews on rebase, etc.
Cheers! 🙏

Unfortunately not yet. I was also wondering if this would be a good candidate for a GitHub "Action". Then I don't have to worry about the hosting part or webhooks getting lost either, although it's been pretty solid to date.

Just to be clear, you are talking about a GitHub Action on your end, correct? As opposed to having users set Actions on each repos where Renovate runs.

No, I was meaning self-hosted actions. I don't think actions can be used by apps to serve multiple accounts.

If I understand correctly, that would require us to set them for all our repos, correct? We have a bunch, so that seems like a lot of overhead :/
Also, I do not have access to the admin panels of my org, so we would have to escalate this, which would more than likely lead to trial-errors, etc.
I would love if this was handled by Renovate itself, and we wouldn't have to worry about configuration on our end :)

The only issue with using Actions at the moment is that they are not available for all repos yet (especially within organizations). The team I work with would not be able to use an action at the moment.

I was doing some experimenting to understand how Github dismisses pull requests and what you get for information from the webhook.

It looks like you can fundamentally solve this problem using the pull_request.dismissed event.

The event contains the essential pieces of information to know if a PR was dismissed, by whom what it dismissed, and who was the current approver.

{
  "action": "dismissed",
  "review": {
    "user": {
      "login": "renovate-approve[bot]",
      //...
    },
    //...
  },
  "pull_request": {
    "user": {
      "login": "renovate[bot]",
      //...
    },
    //...
  },
  "sender": {
    "login": "renovate[bot]",
    //...
  }
}

This is an example of a PR dismissal from one of my repos. You can see that renovate-approve[bot] is the review.user.login, the pull_request.user.login is renovate[bot], and the sender (Dismisser of the pull request) is renovate[bot]

I think if you base a re-approval on this set of conditions, it would be reasonably safe.

Thoughts?

@terodox that looks like the right way to detect our dismissed review, however we'd still need to find the "new" commit to re-approve, right? I was wondering if it's simpler to instead just do this:

For every push event (or is it PR updated event?), query the approvals API to see if we already have approved
If we have, then skip it (probably they are not dismissing stale reviews)
If not, then add the approval (again)

If we need the sha for the commit that dismissed the review that is available in the pull_request.head.sha

However, I'm not sure we need the new commit. In the opened event handler that is already in use we only needed the pr number to create a new review.

Am I missing the importance of knowing more information about newest commit?

I was maybe getting mixed up with status checks, which are per-commit.

I've spent about an hour trying to validate this, but I'm coming up short with testing github apps on my local system.

The basic idea is here though:

  app.on('pull_request_review.dismissed', async context => {
    if (
        context.payload.sender.login === 'renovate[bot]' &&
        !context.payload.body.pull_request.body.includes('merge this manually') &&
        context.payload.review.user.login === 'renovate-approve[bot]' &&
        context.payload.pull_request.user.login === 'renovate[bot]'
      ) {
        const params = context.repo();
        params.number = context.payload.number;
        params.event = 'APPROVE';
        return context.github.pullRequests.createReview(params);
      }
  });

I am more than willing to open a PR to the repo adding this, but I have not been able to get probot running locally to validate it.

It will require the additional permission for - pull_request_review and the default_permissions will need to have pull_requests: read added.

Let me know if you'd like a PR that you could then validate.

Thanks for working through this with me 🎉

A PR would be great. Any time I deploy a new version of this I always have to remember how I did it the last time too.. but I'm sure I'll remember it again.

This should now be working. Thanks all for the help

Support dismissed reviews/force pushes