Discussion: consider disabling wheel testing always
yarnabrina opened this issue · comments
For every release, two PR's are created: first to disable testing full test suite and second to revert first. Is there any reason to not remove it permanently?
My main reason for not removing it is:
- I consider it a safety mechanism, reducing the risk of error.
- It introduces a delay between pressing the green button and the actual release
- It runs the full test suite between the green button and the actual release, so will not release if there are genuine (not sporadic) failures.
- in the past, I have skipped this due to sporadic failures, or failures we knew were inconsequential. However, in all cases, I've run through the mandatory pre-release test, which I think improved safety and reliability.
Phrased more negatively, if we remove it permanently, I would be apprehensive of irreversible release actions by accident.
I agree with all the safety comments and it makes total sense. The poiof this issue is just noting the skips recently.
To ensure a manual approval or wait period, you can consider this action, or the environment option.
https://github.com/marketplace/actions/manual-workflow-approval
https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments
Yes, good idea.
Though, instead of manual approval, how about a simple time delay? Given history, I am not sure if it is feasible to insist on reviews at every release, in the sense that these might not materialize and then things get blocked through inaction.
That's probably a lot easier, you can replace the test call with sleep 1h and that's it.