Doesn't review on updates
chrisns opened this issue · comments
with the Dismiss stale pull request approvals when new commits are pushed setting enabled on branch protection then the approve-bot doesn't re-approve on subsequent commits
what this means in the real world is if renovatebot creates 5 prs, 1 will get approved and merged, the others will auto rebase or merge commit, but then approve-bot doesn't approve it, so i've now got 4 prs I need to manually approve.
Hope that makes sense?
It would appear that this: https://github.com/renovatebot/renovate-approve-bot/blob/main/index.js#L61
isn't happening, for example:
DaftDoris/newsdesk#447 (comment)
Please create a public reproduction repo you can demonstrate the problem on and leave the PR open
@chrisns thanks for the reproduction. The approve boats are not approving due to this text: Automerge: Disabled due to failing status checks.
If status checks are failing, aren't you going to need manual intervention to merge anyway, or are these optional status checks and you were hoping for it to automerge regardless?
@rarkins you're quite right, sorry that's a poor example
this one is better: DaftDoris/newsdesk#452
We need a more stable repository to test against. The current one is a real on and getting too many commits. For example:
We can see that Renovate's first force push dismissed the reviews, which should have triggered more, but then there's a second force push after that (and more after that).
Studying the edits we can see that there was a period where automerge was disabled, which is presumably why the bots didn't re-approve their dismissed reviews:
Is it 'right/necessary' that pull_request_review.dismissed is tightly coupled to the check results at the time that the review is dismissed, those can can change or be async? or even tightly coupled to that auto-merging is enabled?
I'd expect the branch protection config to keep the branch safe from checks failing etc
Therefore would it be reasonable to ask for https://github.com/renovatebot/renovate-approve-bot/blob/main/index.js#L66 to be removed? (happy to raise the PR)
I would prefer to find another way (such as a different event to listen to). It's preferred that we don't approve non automerging PRs so as to reduce "noise"
if that was configurable somehow that'd be awesome, seems like it'd be hard and require calling back to the API to get more context than you'd get in the payload from the checks which may be a disproportionate mitigation to the additional noise factor?
I guess for now I can do a super ugly hack workaround by setting the prfooter to **Automerge**: Enabled in my renovate config
https://docs.renovatebot.com/configuration-options/#prfooter in order to trick the bot
Yeah, right now we don't have to any API calls to know if we should approve or not. But I don't know if perhaps the pull_request>edited event could work too? That should at least be triggered any time the bot edits the PR body, e.g. to set it back to "Automerge: enabled"
yeah that could work, will raise a PR, probably won't cause any harm but do you have to means to test this (test version of the app install) we could use to check before rolling out to the wider userbase?
I just remembered something else important. The reason we went for approving when reviews were dismissed and not whenever an update was made was because:
- If we keep reviewing, they stack on top of each other until the point where no more reviews are allowed, and
- We otherwise would need to query using the API each time to see if our review was dismissed, etc. I think this was avoided mostly due to complexity reasons and not performance though
