Move the manual steps of our release process to cmrel commands

[Maël Valais]

As described in "deliverable 1", we need to rework cmrel so that cmrel is the single command we need for cutting a new release.

Here is the list of manual steps that we want integrated into cmrel:

1. the release notes (currently done by release-notes wrangling that we have to do in our release process)
2. updating or creating the release branch (currently done manually, tricky step),
3. find a way to "chain" cmrel stage, cmrel publish, and cmrel publish --nomock) without having to copy-paste manually the "release name" between each step

This issue is about automating (1), (2), and (3), and to make (4) as easy as possible e.g. just uncheck the "Draft" option in the GitHub release.

Progress as of now:

• create cmrel update-release-branch: ETA 3 days work, 20% done (#36) see #31 (comment)
• create release-notes: ETA 1 day work, 0% done
• automate passing the "release name" between the stage, publish, and publish --nomock steps: ETA 1 day work, 0% done

[Maël Valais]

@jakexks I'm not too sure about the usefulness of cmrel update-release-branch. Right now (#36), the command

cmrel update-release-branch --branch release-1.4

does not know about the "code freeze". update-release-branch must not be run during a code freeze, since merged PRs must be cherry-picked during that period.

I think --branch release-1.4 isn't good, I should rather go with

cmrel update-release-branch-for v1.3.1

Behavior:

• initial and sebsequent alpha release v1.3.0-alpha.0, v1.3.0-alpha.1: create the release-1.3 branch if it does not exist, merge --ff-only master and push release-1.3
• beta release v1.3.0-beta.0: do nothing
• final release v1.3.0: do nothing
• patch release v1.3.1: do nothing

[Maël Valais]

Here is an analysis of the current release process:

Current release process

1. Update (push) the release branch.
   • This steps depends on whether this is a subsequent alpha or not, and it would make sense to just have to give the "release version" and cmrel would perform the correct "update" depending on whether this is a subsequent alpha or not.
2. Create release notes as a file on the disk and share it with the team for inspection by running the command:

   START_REV="v1.4.0" END_REV="release-1.5" BRANCH="release-1.5" RELEASE_VERSION="1.5.0" \
     release-notes --debug --repo-path=cert-manager --org=jetstack --repo=cert-manager \
       --required-author=jetstack-bot --output release-notes.md
   

   • It is hard to get the correct incantation,
   • We almost always forget that the END_REV should be the child of e.g. tag v1.2.0.
   • The file is created on disk. We should rather have it uploaded to a GitHub Release draft or to https://hackmd.io so that we can collaborate on editing the notes.
3. Run the following command and wait to 12 minutes:

   cmrel stage --branch=release-1.2 --release-version=v1.2.0
   

   • Although there are only two arguments, it would be nice if we just had to give the "release version" (e.g. v1.2.0) and the branch is guessed from that.
   • Lot of logs, which makes it overwhelming (I don't know what I should be paying attention to with all these logs).
   • We don't know when the command ends, it would be nice to have some form of progress indicator (e.g. 10 minutes remaining).
4. Pick up the last part of the gs:// URL that was printed by the previous command and use it to run the command (takes about 2 minutes):

   cmrel publish --release-name=v1.2.0-791347b3c9f5406a0b1fc796cc677bd0ddd1266a

   • Finding the "release notes" among the logs from the previous command isn't "obvious" since the logs don't mention "release name"; you have to "guess" that the last segment of the bucket URL is the release name.
   • Again, a lot of logs coming out from this command, and I don't know which information is important while running the command.
5. Re-run the previous command with --nomock:

   cmrel publish --release-name=v1.2.0-791347b3c9f5406a0b1fc796cc677bd0ddd1266a --nomock

   • What is the purpose of this? I would either suggest to remove the "mock" step, or to run them one after the other (if the mock fails, the "nomock" is not run).
   • The publish command should also create a new release branch when we publish a final release.

Timing:

Command	Time to run
release-notes	1 minute
cmrel stage	12 minutes
cmrel publish	2 minutes
cmrel publish --nomock	6 minutes