Windows machines in the CI are down
StefanStojanovic opened this issue · comments
Almost all of the Windows machines in the CI are down. From what I saw on a few of them, this is because of the Temurin17 update. I'll go through all of them and bring them back up. After this is done I'll close this issue and open a ticket for fixing a root cause (Temurin17 update).
Most of the machines are restored. Only a few Win2022_VS2022 are left, because I cannot access them through RDP (they are locked out), but I'll try periodically and eventually will get them too. Leaving this issue open until then.
What's doing the update? Temurin doesn't have any in-built upgrade facility.
Do you know what the error/failure was? Was it the package manager we're using that failed to update or an issue with the Temurin build itself?
It's handled by Chocolatey. You can see an example failure here: https://ci.nodejs.org/view/All/job/windows-update-reboot/2534/label=test-azure_msft-win2022_vs2019-x64-1/console
Interesting ... Not clear what might have gone wrong there, but I've just tried an clean install of an earlier x64 MSI and been able to update it with the new MSI (Not using chocolatey) without issues (albeit from the UI)
All of the machines are back online. Now regarding my investigation - the windows-update-reboot job is to be blamed (as @targos already pointed out). What I saw after logging in is Terminate batch job (Y/N)? in the Jenkins CMD window. Luckily, Temurin is not updated that often, but we should definitely find a way to fix this, as once it updates it breaks the entire Windows CI. I'll open another issue for that and the discussion can continue there.
FYI Temurin 17 will get an update once every 3 months (It's a regular cadence from the upstream openjdk project).
Every so often there will be an intermediate release for a critical bug fix or security patch in addition to that but it doesn't happen too often.