The Ubuntu 18.04 Actions runner image will begin deprecation on 2022/08/08 and will be fully unsupported by 2023/04/03

Question

The Ubuntu 18.04 Actions runner image will begin deprecation on 2022/08/08 and will be fully unsupported by 2023/04/03

miketimofeev opened this issue 2 years ago · comments

Breaking changes

We have started the deprecation process for Ubuntu 18.04. While the image is being deprecated, You may experience longer queue times during peak usage hours. Deprecation will begin on 2022/08/08 and the image will be fully unsupported by 2023/04/03

To raise awareness of the upcoming removal, we will temporarily fail jobs using Ubuntu 18.04. Builds that are scheduled to run during the brownout periods will fail. The brownouts are scheduled for the following dates and times:

GitHub Actions\Azure DevOps::

October 3, 12:00 UTC – October 3, 14:00 UTC
October 18, 14:00 UTC – October 18, 16:00 UTC
November 15, 18:00 UTC – November 15, 20:00 UTC
November 30, 20:00 UTC – November 30, 22:00 UTC
December 15, 20:00 UTC – December 16 00:00 UTC
January 5, 10.00 UTC – January 5, 14.00 UTC
January 13, 12.00 UTC – January 13, 16.00 UTC
January 18, 14.00 UTC – January 18, 18.00 UTC
January 24, 16.00 UTC – January 24, 20.00 UTC
February 7, 16.00 UTC – February 7, 22.00 UTC
February 21, 10.00 UTC – February 21, 22.00 UTC
March 6, 00.00 UTC – March 7, 00.00 UTC
March 13, 00.00 UTC – March 14, 00.00 UTC
March 21, 00.00 UTC – March 22, 00.00 UTC
March 28, 00.00 UTC – March 29, 00.00 UTC

Target date

April 1st, 2023

The motivation for the changes

We maintain the latest two stable versions of any given OS version. Ubuntu 22.04 is going GA on 8/8/22 thus we start deprecating the oldest image.

Possible impact

Workflows using the ubuntu-18.04 image label should be updated to ubuntu-latest, ubuntu-20.04, or ubuntu-22.04

Platforms affected

Azure DevOps
GitHub Actions

Virtual environments affected

Mitigation ways

N\A

Tinus commented 2 years ago

Tlustoch

gibz commented 2 years ago

🤡

Abduljakulsalsal commented a year ago

✌️

Willem Mobach · Answer 1 · Mon Aug 08 2022 18:40:11 GMT+0800 (China Standard Time)

Don't get me wrong, I fully understand that you're deprecating 18.04 - but incorporating brownout periods to raise awareness?

If my team needs to get a build out for a critical bugfix on the 22nd of August... I'm just going to have to wait for 4 hours? If my understanding of that is true, I find that very odd behavior in an enterprise environment. Some businesses actually have SLAs with their clients. Would kindly request the team to find another way to "raise awareness". I came here because of the red banner when using the image, that worked fine as well and is a lot less invasive.

You're giving us two weeks to migrate away before brownouts start. Migrating is an action that requires testing and validation.

francis · Answer 2 · Tue Aug 09 2022 01:05:57 GMT+0800 (China Standard Time)

Does this impact hosted agents only, or does this also mean the agent tooling/infra (for self hosted pools, but also for 1ES pools for internal folks) will become EOL for ubuntu18? My team still needs to keep shipping ubuntu18 versions of our product for years.

Jakub Panek · Answer 3 · Tue Aug 09 2022 06:13:26 GMT+0800 (China Standard Time)

@fbrosseau it's hosted agent only

Erik Smistad · Answer 4 · Thu Aug 11 2022 18:58:20 GMT+0800 (China Standard Time)

I am sad to see this removal of ubuntu-18.04 and hope you will reconsider.
I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases.
Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.

Mikhail Koliada · Answer 5 · Thu Aug 11 2022 19:18:50 GMT+0800 (China Standard Time)

I am sad to see this removal of ubuntu-18.04 and hope you will reconsider. I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases. Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.

@smistad as per the policy we only support the two latest LTS

Jakub Panek · Answer 6 · Thu Aug 11 2022 19:20:38 GMT+0800 (China Standard Time)

I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases.
Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.

You can use container: option to build with ubuntu:18.04 image or statically link binaries (or install Ubuntu 18.04 sysroot on new runners and build with that).

Erik Smistad · Answer 7 · Thu Aug 11 2022 19:30:11 GMT+0800 (China Standard Time)

I use github to build C++ software on ubuntu-18.04, essentially making it possible to run it on 18.04 and newer releases.
Due to the lack of backward compatability when compiling with the C++ libraries on linux (GLIBC), binaries compiled on 20.04 will not work on 18.04.

You can use container: option to build with ubuntu:18.04 image or statically link binaries.

Thanks for the tip @panekj

Olly Betts · Answer 8 · Fri Aug 12 2022 10:02:53 GMT+0800 (China Standard Time)

Deprecation will begin on 8/8/2022 and the image will be fully unsupported by 12/1/2022

Should be 12/1/2023 or is it a US format date?

Aaron Franke · Answer 9 · Fri Aug 12 2022 10:42:22 GMT+0800 (China Standard Time)

@smistad as per the policy we only support the two latest LTS

@mikhailkoliada Is there any way we can convince GitHub to change this policy to the latest three LTS releases? We actually switched from 20.04 to 18.04 a few months ago because we found it worked better for our use cases.

deathaxe · Answer 10 · Thu Aug 18 2022 01:57:22 GMT+0800 (China Standard Time)

What about projects which still need python 3.3 support for a reasonable time?

see: https://github.com/wbond/package_control/runs/7884696802?check_suite_focus=true

You must not simply break existing workflows!!!!

Scott Shambaugh · Answer 11 · Fri Aug 19 2022 10:04:51 GMT+0800 (China Standard Time)

Ubuntu 18.04 does not reach the end of its standard support phase until April 2023. If the policy here is to support the last two LTS releases, I would suggest it be amended to align dropping support when Ubuntu does.

And another vote for 2 weeks before the brownouts start being very short.

probonopd · Answer 12 · Sun Aug 21 2022 23:05:06 GMT+0800 (China Standard Time)

Ubuntu 18.04 does not reach the end of its standard support phase until April 2023. The AppImage project is relying on it to be available on GitHub Actions for the whole timeframe doring which it is supported.

Dirkjan Bussink · Answer 13 · Mon Aug 22 2022 21:25:02 GMT+0800 (China Standard Time)

There seems to be quite a discovery issue with the brown out. I hadn't seen this issue originally which isn't the biggest deal, but seeing CI jobs fail with no context whatsoever is very confusing.

All I saw was a "This check was cancelled" with no context whatsoever about why it was cancelled. No message that there's a brown out, no link to this issue etc.

Somehow I was able to navigate to a different page of a status check that finally showed that information. Is it possible to make this more visible and prominent so it can be found easier?

It seems to not be visible if you click "Details" from a pull request. Only when manually navigating from the "Actions" tab can this be found.

Dan Fairs · Answer 14 · Mon Aug 22 2022 21:27:09 GMT+0800 (China Standard Time)

So yeah, we've just found out that 18.04 is deprecated. Considering I just upgraded to Enterprise at the cost of the order of thousands of dollars a month, I'm not massively pleased to see you just breaking our whole company's jobs for four hours.

Couldn't this be done in a different way? A banner on jobs that use the image? An email to org admins? It's pretty bad behaviour.

Jakub Panek · Answer 15 · Mon Aug 22 2022 21:35:14 GMT+0800 (China Standard Time)

A banner on jobs that use the image?

Dan Fairs · Answer 16 · Mon Aug 22 2022 21:38:36 GMT+0800 (China Standard Time)

A banner on jobs that use the image?

What can I say? A whole company of 20 people didn't notice it.

Jakub Panek · Answer 17 · Mon Aug 22 2022 21:41:46 GMT+0800 (China Standard Time)

There is also https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22

Dan Fairs · Answer 18 · Mon Aug 22 2022 21:45:48 GMT+0800 (China Standard Time)

There is also https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22

I'm sure there is. But we pay for this. You (as in GH, not you personally) need to be proactive about reaching out to customers when you're planning to break something. The rest of Microsoft is pretty good at this.

Edvard Hübinette · Answer 19 · Mon Aug 22 2022 21:52:11 GMT+0800 (China Standard Time)

This brownout scheme is the worst way to "raise awareness for deprecation" and has put a dark tint over our otherwise happy relationship with GitHub. Who figured that this would be a good solution? We sure disagree.

Ovidiu Calbajos · Answer 20 · Mon Aug 22 2022 21:59:23 GMT+0800 (China Standard Time)

Why not send some emails? It's easier than screwing peoples builds :)

Allan MacGregor · Answer 21 · Mon Aug 22 2022 22:34:58 GMT+0800 (China Standard Time)

@miketimofeev Brownout is pretty extreme, you are getting in the way of people shipping and being able to work. And stopping for service we are paying for.

This is huge loss of trust, and pushing us to consider alternatives.

Steve-Glass · Answer 22 · Mon Aug 22 2022 22:56:31 GMT+0800 (China Standard Time)

👋 Thank you all for expressing your concerns with the deprecation schedule and feedback on preferred availability. We will revise the deprecation schedule and are will immediately stop the brownouts.

j4mb0ti · Answer 23 · Mon Aug 22 2022 22:56:37 GMT+0800 (China Standard Time)

We are working on a complex migration with a pretty tight deadline (tomorrow). We have noticed the deprecation and planned for it accordingly. Running into a random brownout is the worst thing any tool could do to us at the moment.
Yes, we are aware of the deprecation. Yes, we have planned to make the necessary changes when we are able to. You are just adding insult to an injury - not raising awareness. Much disappointment. I thought microsoft would have learned from its passed mistakes - but apparently we are still in the Windows SE area. That's the kind of awareness you are raising. And now I am pretty sure I'll keep my other projects over at GitLab.

Matt Muller · Answer 24 · Mon Aug 22 2022 23:16:50 GMT+0800 (China Standard Time)

This is impacting AWS SDK For Ruby CI - I agree with others that this is a poor way to raise awareness, as this is the first time I've heard of this campaign. We can make steps to migrate.

rtek · Answer 25 · Mon Aug 22 2022 23:35:21 GMT+0800 (China Standard Time)

Please come up with a more reasonable way to handle deprecation of the runners.

Consider how AWS handles lambda runtime deprecations as an example. They distribute emails when old runtimes are being used. You have access to all the workflows so you know whos using them.

Ivo Jansch · Answer 26 · Tue Aug 23 2022 12:46:14 GMT+0800 (China Standard Time)

With a three week notice period, introducing these brownouts is like pulling the deprecation date forward. Because you can’t rely on a production system that breaks deployments every week for 6 hours. Not being able to perform an action during this time is Ill-advised and a serious risk for anyone relying on GitHub actions for production deployments.

It’s like putting a house on fire to raise awareness that playing with matches is dangerous.

See https://www.linkedin.com/posts/jansch_github-cicd-activity-6967561507354120192-084T?utm_source=share&utm_medium=member_ios for how this lead to a serious production incident at one of our customers when we needed to roll back a release because of a bug.

At least make this a setting so we can only do this
‘Forced awareness’ on test/acceptance builds. A feature like this on production is just killing reliability or GitHub actions for serious production work.

Matt Lewis · Answer 27 · Tue Aug 23 2022 19:30:29 GMT+0800 (China Standard Time)

We got hit by this brownout yesterday but nothing told us our jobs were getting cancelled because of the brownout, I found this issue purely by chance. If you're going to do something like this you really need to give an obvious error message and explain why jobs are being cancelled, just seeing the job being cancelled made me think there was an outage. Even our github account manager wasn't aware of this brownout that was happening.

fcjz · Answer 28 · Tue Aug 23 2022 21:12:35 GMT+0800 (China Standard Time)

Also, last night, I also experienced this brownout , but there is no explanation, I feel a little baffling.

Too sloppy.

Martin Galvan · Answer 29 · Fri Aug 26 2022 21:56:48 GMT+0800 (China Standard Time)

Don't get me wrong, I fully understand that you're deprecating 18.04 - but incorporating brownout periods to raise awareness?

If my team needs to get a build out for a critical bugfix on the 22nd of August... I'm just going to have to wait for 4 hours? If my understanding of that is true, I find that very odd behavior in an enterprise environment. Some businesses actually have SLAs with their clients. Would kindly request the team to find another way to "raise awareness". I came here because of the red banner when using the image, that worked fine as well and is a lot less invasive.

You're giving us two weeks to migrate away before brownouts start. Migrating is an action that requires testing and validation.

We had the exact same problem when windows-2017 was deprecated, and it was extremely painful to have this transition forced unto us.

IMHO this one is particularly bad since there are legitimate reasons to build on older versions of Ubuntu; namely, so that the linker won't pull in symbols from newer versions of glibc which the customer's systems don't have.

Mikhail Timofeev · Answer 30 · Wed Aug 31 2022 00:28:49 GMT+0800 (China Standard Time)

The deprecation date has shifted to March 1st, 2023 to give more time to adapt the workflows to Ubuntu20 or Ubuntu22.
The brownout schedule is also adjusted and now has shortened periods.

Dirkjan Bussink · Answer 31 · Wed Aug 31 2022 01:19:36 GMT+0800 (China Standard Time)

The deprecation date has shifted to March 1st, 2023 to give more time to adapt the workflows to Ubuntu20 or Ubuntu22.
The brownout schedule is also adjusted and now has shortened periods.

@miketimofeev Is there also a plan to make it much more obvious in the UI that the brownout is happening? The lack of any information in the view linked from pull requests made the user experience significantly worse here.

Mikhail Timofeev · Answer 32 · Wed Aug 31 2022 17:32:54 GMT+0800 (China Standard Time)

The deprecation date has shifted to March 1st, 2023 to give more time to adapt the workflows to Ubuntu20 or Ubuntu22.
The brownout schedule is also adjusted and now has shortened periods.

@miketimofeev Is there also a plan to make it much more obvious in the UI that the brownout is happening? The lack of any information in the view linked from pull requests made the user experience significantly worse here.

it's on our radar but I'm afraid we can't guarantee that better UI notifications will be implemented during this deprecation.

Mikhail Timofeev · Answer 33 · Fri Sep 02 2022 14:29:52 GMT+0800 (China Standard Time)

We have decided to prolong the deprecation until the official end of life — April 2023. Brownouts schedule has been adjusted as well to reflect the prolonged deprecation

JimmyBanks · Answer 34 · Wed Sep 07 2022 22:03:16 GMT+0800 (China Standard Time)

We have decided to prolong the deprecation until the official end of life — April 2023. Brownouts schedule has been adjusted as well to reflect the prolonged deprecation

Our project is using 'ubuntu-latest', but the pipelines continually default to 18.04 giving us this alert/error notice.

Mikhail Timofeev · Answer 35 · Wed Sep 07 2022 22:33:22 GMT+0800 (China Standard Time)

@JimmyBanks could you share a few links to your pipelines with these warning and the screenshot of the Initialize job step with the runner image?

JimmyBanks · Answer 36 · Thu Sep 08 2022 03:14:46 GMT+0800 (China Standard Time)

@JimmyBanks could you share a few links to your pipelines with these warning and the screenshot of the Initialize job step with the runner image?

Turns out this was a mistake on our end and the pipeline was set to 18.04, not latest. We mixed up two different projects settings, and assumed both were updated at the same time to latest.

Sorry for wasting your time

René Hermenau · Answer 37 · Thu Sep 08 2022 19:18:59 GMT+0800 (China Standard Time)

To raise awareness of the upcoming removal, we will temporarily fail jobs using Ubuntu 18.04.

This will explain why our jobs suddenly fail without doing any modifications.
We are paying you a lot of money for CI minutes, so you should inform your customers properly in advance, keep the deprecated services running until expiration date instead of making your clients tests fail and rising their costs for extra CI minutes and development time.

Sean McManus · Answer 38 · Tue Sep 20 2022 13:53:38 GMT+0800 (China Standard Time)

Our data shows Ubuntu 18 usage is still close to Ubuntu 22 usage. Can you delay the deprecation/removal until more users have migrated off Ubuntu 18? The support for only 2 OS versions seems arbitrary and not driven by usage data?

Mikhail Timofeev · Answer 39 · Tue Sep 20 2022 14:10:08 GMT+0800 (China Standard Time)

@sean-mcmanus we will support Ubuntu 18 until it reaches its end of life in April 2023

awaters-pa · Answer 40 · Fri Oct 14 2022 20:54:55 GMT+0800 (China Standard Time)

This is happening today and today is not on your list....any advice? We are very aware of the deprecation and don't need our productivity on the needed changes to be blocked by an artificial hindrance.

Patrick Prakash · Answer 41 · Fri Oct 14 2022 21:01:41 GMT+0800 (China Standard Time)

This incident occurred while running CI pipeline of our platform. At least they could have mentioned this before the scheduled brownout. Any way nice way to change deprecated dependencies!

Henrik Sylvester Oddergaard · Answer 42 · Fri Oct 14 2022 21:07:04 GMT+0800 (China Standard Time)

Yeah wtf, this was not scheduled and on a friday... I was about to go home with a nice bugfixed shipped out....

GeoFro · Answer 43 · Fri Oct 14 2022 21:25:06 GMT+0800 (China Standard Time)

October 14th is not on your brownout list.

awaters-pa · Answer 44 · Fri Oct 14 2022 22:06:23 GMT+0800 (China Standard Time)

Pretty slimy to update the schedule after you get complaints. This brownout should have been cancelled and not spring on everyone for a six hour window. This is completely irresponsible on your part @github

Oleg Karasik · Answer 45 · Fri Oct 14 2022 22:20:52 GMT+0800 (China Standard Time)

The best moment here is that jobs are get canceled and ... no notifications are send by email. We have just lost a few hours before realised that our build is not running and not propagating to environments. Thanks you very much, you increased my awareness of other platforms.

Alessandro Muntoni · Answer 46 · Fri Oct 14 2022 22:49:56 GMT+0800 (China Standard Time)

Could you please at least change the error message and update to the right removal date?? Is it December 1st or April 1st?
I hope you can confirm that it will be April 1st, because EOL of Ubuntu 18.04 is April 2023.

This is totally unprofessional.

Ollie Lowson · Answer 47 · Fri Oct 14 2022 23:04:13 GMT+0800 (China Standard Time)

For posterity, @ddobranic / @github.

Before:
https://web.archive.org/web/20221012145059/https://github.com/actions/runner-images/issues/6002

After:

awaters-pa · Answer 48 · Fri Oct 14 2022 23:05:14 GMT+0800 (China Standard Time)

And the blog still isn't updated... 🙄
https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22/

Mikhail Timofeev · Answer 49 · Fri Oct 14 2022 23:07:36 GMT+0800 (China Standard Time)

@ollielowson-wcbs sorry, I'm not part of the team anymore and am not aware of the changes to the schedule.
Please ping @ddobranic for clarification

Dusko Dobranic · Answer 50 · Fri Oct 14 2022 23:40:10 GMT+0800 (China Standard Time)

@awaters-pa sorry, my bad. reverted to original dates.

awaters-pa · Answer 51 · Fri Oct 14 2022 23:57:24 GMT+0800 (China Standard Time)

@awaters-pa sorry, my bad. reverted to original dates.

So when should we expect this unexpected brownout to end? @ddobranic

redaER7 · Answer 52 · Sat Oct 15 2022 00:01:26 GMT+0800 (China Standard Time)

brownouts are starting...
of course, with no notifications or alert...

Piers Karsenbarg · Answer 53 · Sat Oct 15 2022 00:02:15 GMT+0800 (China Standard Time)

Information on brownouts here: #6002

Also here: https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22/

awaters-pa · Answer 54 · Sat Oct 15 2022 00:03:17 GMT+0800 (China Standard Time)

Information on brownouts here: #6002

Why link back to this issue? The dates are wrong. They were changed and then changed back.

Piers Karsenbarg · Answer 55 · Sat Oct 15 2022 00:04:50 GMT+0800 (China Standard Time)

Because when I first saw the link to this issue it said that the brownouts were due today 🤷

mihaylovg · Answer 56 · Sat Oct 15 2022 02:36:42 GMT+0800 (China Standard Time)

@ddobranic Only registered to let you know what happened today was not professional at all.
My client was not able to deploy changes to production so they are not happy too.

manx · Answer 57 · Tue Oct 25 2022 14:24:37 GMT+0800 (China Standard Time)

Please use a non-broken date format in the issue title: https://en.wikipedia.org/wiki/ISO_8601

Arthur Baars · Answer 58 · Tue Nov 01 2022 19:15:55 GMT+0800 (China Standard Time)

Out of curiosity is 4/1/2023 an English or an American date? Could you update the title to be unambiguous, please?

aalbacetef · Answer 59 · Wed Nov 16 2022 03:38:43 GMT+0800 (China Standard Time)

Don't get me wrong, I fully understand that you're deprecating 18.04 - but incorporating brownout periods to raise awareness?

If my team needs to get a build out for a critical bugfix on the 22nd of August... I'm just going to have to wait for 4 hours? If my understanding of that is true, I find that very odd behavior in an enterprise environment. Some businesses actually have SLAs with their clients. Would kindly request the team to find another way to "raise awareness". I came here because of the red banner when using the image, that worked fine as well and is a lot less invasive.

You're giving us two weeks to migrate away before brownouts start. Migrating is an action that requires testing and validation.

Literally just ran into this. CI started failing due to the brownout while working on a bugfix.

Boris Dalstein · Answer 60 · Wed Nov 16 2022 03:55:24 GMT+0800 (China Standard Time)

Pretty please, GitHub, would it be possible to stop with this "brownout periods" insanity? Some people are trying to work.

In order to raise awareness, I would recommend sending emails. As is, Ubuntu 18.04 is still supported and breaking builds is poor practice. At the very least add a button "Yes, I'm aware of the deprecation, please always run my Ubuntu 18.04 builds anyway from now on".

Note that the reason we're still using Ubuntu 18.04 is not that our organization is slow to migrate to newer versions. The reason is that we need to generate AppImages, and these AppImages must be generated on Ubuntu 18.04 until at least April 2023, and ideally even after that. Our build scripts work perfectly fine onubuntu-latest, but we have to build them on Ubuntu 18.04, otherwise the built binaries would link to versions of libc which are too new.

phlax · Answer 61 · Wed Nov 16 2022 04:04:41 GMT+0800 (China Standard Time)

sorry - but this has to take the prize for unnecessarily alienating your customers

Anthony Scott · Answer 62 · Wed Nov 16 2022 04:15:26 GMT+0800 (China Standard Time)

Can you at least adhere to your own timelines?

Ross van Zyl · Answer 63 · Wed Nov 16 2022 04:27:53 GMT+0800 (China Standard Time)

Can you at least adhere to your own timelines?

Agreed.
At the time of this comment, the original post states November 15, 18:00 UTC – November 15, 20:00 UTC but this brownout seemed to start at 20:00 UTC

Boris Dalstein · Answer 64 · Wed Nov 16 2022 05:05:28 GMT+0800 (China Standard Time)

It's 21:02 UTC, builds are still failing. Oh, and they did fail already soon after 18:00 UTC. The only reasonable course of action for GitHub is to cancel all brownouts periods until an opt-out mechanism is implemented, this is simply not acceptable.

Steffen Jaeckel · Answer 65 · Wed Nov 16 2022 16:47:01 GMT+0800 (China Standard Time)

IMO your decision logic on the helpfulness of these brownout tests is flawed.

The assumption that this leads to fewer projects breaking sometime in the future when a specific runner is deprecated won't work, as this brownout mechanism is only triggered for "very active" projects. If a project is e.g. feature complete and has not many rebuilds, chances are high that this will not help at all and builds will only break sometime in the future after the runner was deprecated.

Instead you're hurting all projects that maybe have a good reason to still build against "soon to be deprecated runners" as already indicated by many users.

[...] Ubuntu 18.04 is still supported and breaking builds is poor practice. At the very least add a button "Yes, I'm aware of the deprecation, please always run my Ubuntu 18.04 builds anyway from now on".

^ this

The only reasonable course of action for GitHub is to cancel all brownouts periods until an opt-out mechanism is implemented, this is simply not acceptable.

^ and this

hosseinmirzae · Answer 66 · Wed Nov 16 2022 22:25:19 GMT+0800 (China Standard Time)

Breaking changes

We have started the deprecation process for Ubuntu 18.04. While the image is being deprecated, You may experience longer queue times during peak usage hours. Deprecation will begin on 2022/08/08 and the image will be fully unsupported by 2023/04/01

To raise awareness of the upcoming removal, we will temporarily fail jobs using Ubuntu 18.04. Builds that are scheduled to run during the brownout periods will fail. The brownouts are scheduled for the following dates and times:

GitHub Actions\Azure DevOps::

October 3, 12:00 UTC – October 3, 14:00 UTC

October 18, 14:00 UTC – October 18, 16:00 UTC

November 15, 18:00 UTC – November 15, 20:00 UTC

November 30, 20:00 UTC – November 30, 22:00 UTC

December 15, 20:00 UTC - December 16 00:00 UTC

January 5, 10.00 UTC - January 5, 14.00 UTC

January 13, 12.00 UTC - January 13, 16.00 UTC

January 18, 14.00 UTC - January 18, 18.00 UTC

January 24, 16.00 UTC - January 24, 20.00 UTC

February 1, 18.00 UTC - February 1, 22.00 UTC

February 7, 16.00 UTC - February 7, 22.00 UTC

February 13, 14.00 UTC - February 13, 22.00 UTC

February 21, 10.00 UTC - February 21, 22.00 UTC

February 28, 10.00 UTC - February 28, 22.00 UTC

March 6, 00.00 UTC - March 7, 00.00 UTC

March 13, 00.00 UTC - March 14, 00.00 UTC

March 21, 00.00 UTC - March 22, 00.00 UTC

Target date

April 1st, 2023

The motivation for the changes

We maintain the latest two stable versions of any given OS version. Ubuntu 22.04 is going GA on 8/8/22 thus we start deprecating the oldest image.

Possible impact

Workflows using the ubuntu-18.04 image label should be updated to ubuntu-latest, ubuntu-20.04, or ubuntu-22.04

Platforms affected

Azure DevOps

GitHub Actions

Virtual environments affected

Ubuntu 18.04

Ubuntu 20.04

Ubuntu 22.04

macOS 10.15

macOS 11

macOS 12

Windows Server 2019

Windows Server 2022

Mitigation ways

N\A

gh pr checkout 81

Boris Dalstein · Answer 67 · Fri Nov 18 2022 19:23:55 GMT+0800 (China Standard Time)

For anyone interested, here is a workaround to keep being able to run actions on Ubuntu 18.04 during the brownout periods, or after the deprecation. The idea is to run the action within a Ubuntu 18.04 Docker container, within a ubuntu-latest GitHub self-hosted runner.

Here is a minimal example compiling a C++/CMake project (see ubuntu-18.04-docker-minimal.yml for a live working example).

name: Ubuntu 18.04 (Docker Minimal)
on: [push, pull_request]
jobs:
  build:
    runs-on: ubuntu-latest
    container: ubuntu:18.04
    steps:

    - uses: actions/checkout@v2

    - name: Install Dependencies
      run: |
        apt-get update
        apt-get install -y cmake build-essential

    - name: Build
      working-directory: ${{github.workspace}}
      run: |
        mkdir build
        cd build
        cmake ..
        cmake --build .

Note that actions/checkout@v2 requires git 2.18+ to "properly work", and the default version from the ubuntu:18.04 Docker image is only 2.17. So unless you manually install a newer git, actions/checkout@v2 will simply download the code as is, instead of doing a git clone, and you won't have access to the repo history. In order to install a newer git and get a real git clone, you add the following before the actions/checkout@v2 step:

    - name: Install and Configure Recent Git
      run: |
        apt-get update
        apt-get install -y software-properties-common
        add-apt-repository ppa:git-core/ppa
        apt-get update
        apt-get install -y git
        git config --global --add safe.directory `pwd` # Fix "fatal: detected dubious ownership in repository at '/__w/<user>/<repo>'""

Also, you probably want a newer version of CMake (default is 3.10), which you can do with:

    - name: Install Recent CMake
      run: |
        apt-get install -y gpg wget
        wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | tee /etc/apt/trusted.gpg.d/kitware.gpg >/dev/null
        apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 6AF7F09730B3F0A4
        apt-add-repository "deb https://apt.kitware.com/ubuntu/ bionic main"
        apt-get update
        apt-get install -y cmake

I hope this helps.

By the way, @mikhailkoliada @ddobranic , since this comment will soon be buried in the middle of the 700+-folded comments, feel free to copy this workaround (or link to it) in the original message: it will probably frustrate GitHub users less if they see a workaround at the top of this thread. I sure would have loved to see it instead of spending hours getting something to work.

sgorugantu · Answer 68 · Fri Dec 16 2022 07:56:23 GMT+0800 (China Standard Time)

Even though we are using "windows-latest" as our Vm Image, we are still getting error as "This is a scheduled Ubuntu-18.04 brownout" and this is impacting our pipelines.

Kyle Fazzari · Answer 69 · Wed Feb 08 2023 00:50:08 GMT+0800 (China Standard Time)

Interesting strategy to make sure folks were notified! May I suggest, however, a less hair-pulling message on this page? I re-triggered several times and checked the github status before I finally visited the summary page, which actually contained a very helpful message.

phlax · Answer 70 · Wed Feb 08 2023 01:35:38 GMT+0800 (China Standard Time)

as if to emphasize what an incredibly bad idea this is without any kind of opt out you have done this in the middle of an openssl patch release when projects need to run CI to publish their projects

Ali Gajani · Answer 71 · Wed Feb 08 2023 03:54:08 GMT+0800 (China Standard Time)

I do not think it is a good idea to prevent Github actions from running due to a brownout. You could notify us in a different way, or warn us using some alerts, but preventing a critical push due to this issue isn't a pleasing Developer Experience.

Josh Forbes · Answer 72 · Wed Feb 08 2023 04:12:45 GMT+0800 (China Standard Time)

I do not think a "brownout" is a good strategy for this. Quite frustrating to be honest. We don't want to move off of 18.04, our plan is to move to a container before the deadline. However, it would have been nice to continue operations uninterrupted until the date.

Aleksandar Tokarev · Answer 73 · Wed Feb 08 2023 05:09:33 GMT+0800 (China Standard Time)

I agree with the comments above - brownouts are not the best way to do things - it would be better to have notifications or emails sent with warnings instead.

probonopd · Answer 74 · Sun Mar 05 2023 22:28:52 GMT+0800 (China Standard Time)

the image will be fully unsupported by 2023/04/0

Will it become un_supported_ or un_available_? I hope the former, since I'd like to continue building for it.

CALIFAS95 · Answer 75 · Mon Mar 06 2023 09:53:09 GMT+0800 (China Standard Time)

Breaking changes

We have started the deprecation process for Ubuntu 18.04. While the image is being deprecated, You may experience longer queue times during peak usage hours. Deprecation will begin on 2022/08/08 and the image will be fully unsupported by 2023/04/01

To raise awareness of the upcoming removal, we will temporarily fail jobs using Ubuntu 18.04. Builds that are scheduled to run during the brownout periods will fail. The brownouts are scheduled for the following dates and times:

GitHub Actions\Azure DevOps::

October 3, 12:00 UTC – October 3, 14:00 UTC

October 18, 14:00 UTC – October 18, 16:00 UTC

November 15, 18:00 UTC – November 15, 20:00 UTC

November 30, 20:00 UTC – November 30, 22:00 UTC

December 15, 20:00 UTC – December 16 00:00 UTC

January 5, 10.00 UTC – January 5, 14.00 UTC

January 13, 12.00 UTC – January 13, 16.00 UTC

January 18, 14.00 UTC – January 18, 18.00 UTC

January 24, 16.00 UTC – January 24, 20.00 UTC

February 7, 16.00 UTC – February 7, 22.00 UTC

February 21, 10.00 UTC – February 21, 22.00 UTC

March 6, 00.00 UTC – March 7, 00.00 UTC

March 13, 00.00 UTC – March 14, 00.00 UTC

March 21, 00.00 UTC – March 22, 00.00 UTC

March 28, 00.00 UTC – March 29, 00.00 UTC

Target date

April 1st, 2023

The motivation for the changes

We maintain the latest two stable versions of any given OS version. Ubuntu 22.04 is going GA on 8/8/22 thus we start deprecating the oldest image.

Possible impact

Workflows using the ubuntu-18.04 image label should be updated to ubuntu-latest, ubuntu-20.04, or ubuntu-22.04

Platforms affected

Azure DevOps

GitHub Actions

Virtual environments affected

Ubuntu 18.04

Ubuntu 20.04

Ubuntu 22.04

macOS 10.15

macOS 11

macOS 12

Windows Server 2019

Windows Server 2022

Mitigation ways

N\A

dirkf · Answer 76 · Tue Mar 07 2023 04:15:47 GMT+0800 (China Standard Time)

How do we run tests against old versions of packages not offered in later Ubuntu images?

Kyle Fazzari · Answer 77 · Tue Mar 07 2023 04:17:48 GMT+0800 (China Standard Time)

Consider using docker, @dirkf.

GKOLKO · Answer 78 · Tue Mar 07 2023 15:19:35 GMT+0800 (China Standard Time)

Hello, I am facing issue with the new versions of Ubunto (20, 22 and latest)
I have this error each time: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.3.0:resources (default-resources) on project ebx-technip-module: filtering /home/vsts/work/1/s/mdm-technip/ebx-technip-module/src/main/resources/resources.lnk to /home/vsts/work/1/s/mdm-technip/ebx-technip-module/src/main/webapp/WEB-INF/classes/resources.lnk failed with MalformedInputException: Input length = 1 -> [Help 1]
How can I solve this issue?

Mike McCready · Answer 79 · Tue Mar 07 2023 16:03:34 GMT+0800 (China Standard Time)

@GKOLKO

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-resources-plugin:3.3.0:resources
How can I solve this issue?

This does not sound like an error with the GitHub runner. I suggest you search instead on https://stackoverflow.com/.

nahimmedvent · Answer 80 · Fri Mar 24 2023 04:39:12 GMT+0800 (China Standard Time)

Ubuntu 20.04 and 22.04 don't work at all for me, I keep getting the error /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found (required by /home/site/wwwroot/.python_packages/lib/site-packages/pyodbc.cpython-37m-x86_64-linux-gnu.so)
Because these versions run on 2.29 glibc

Mikhail Koliada · Answer 81 · Fri Mar 24 2023 15:02:13 GMT+0800 (China Standard Time)

@nahimmedvent software on Ubuntu 20.04 and 22.04 is compiled with newer version of libc, which makes the software compiled with older version of libs simply incompatible. There is no straightforward way tp solve this problem as it is a bit non-trivial. The easiest solution would be to switch to ubuntu:18.04 in docker.

LAKSHYA SINGH CHAUHAN · Answer 82 · Wed Mar 29 2023 16:55:32 GMT+0800 (China Standard Time)

This will happen on April fools 🤣

Mikhail Koliada · Answer 83 · Mon Apr 03 2023 21:18:12 GMT+0800 (China Standard Time)

Ubuntu-18.04 is deprecated and disabled for all the customers

Eyal Rozenberg · Answer 84 · Mon Apr 17 2023 04:24:26 GMT+0800 (China Standard Time)

Well, you could at least provide a proper notice about Ubuntu 18.04 runners no longer being available, instead of having actions fail mysteriously...