Tagging process
fcabestre opened this issue · comments
The way Erlang images are tagged may induce issues in Erlang/Elixir application releases. Those releases are said to be tied to a very specific version of Erlang, meaning if a release is build with Erlang 21.1 it may not work with Erlang 21.1.1. This is related to the app.rel file listing all the app dependencies, even the OTP ones. For instance, Erlang 21.1 has application ssl-9.0.2, while Erlang 21.1.1 comes with ssl-9.0.3. A release, depending on the ssl application, built with Erlang 21.1 won't work with Erlang 21.1.1.
That said, the erlang:21.1-alpine and erlang:21.1.1-alpine are based on the same Erlang 21.1.1. I've tried to run an application depending on ssl build with Erlang 21.1 in a container based on erlang:21.1 and failed.
IMHO, when a given image is tagged, it's not a good practice to move the tag to another image.
it's just a common practice among many software versions, the x.y is seen same as x.y.0 and supposed to be able upgrade to x.y.1 smoothly and x.y.z ; it is called Semantic Versioning,
under this Versioning scheme, it implies smooth upgrade, so the x.y can keep updating to alias same content as last x.y.z so that it's a common practice among many other docker images
however, it seems that the OTP release numbers (and the bundled erlang app versions) are not following the Semantic Versioning:
from https://github.com/erlang/otp/releases/tag/OTP-21.1.1 it's bundling the ssl-9.0.3
and http://erlang.org/download/otp_src_21.1.readme is bundling ssl-9.0.2
the problem is these two ssl versions are not compatible, not upgradable
I would like to know some points from upstream, to check if this ssl-9.0.3 is the only exception, or upstream does not like Semantic Versioning at all
the problem is these two ssl versions are not compatible, not upgradable
What is it about this ssl version that makes it not compatible?
If you check https://github.com/erlang/otp/blob/master/otp_versions.table you'll see that otp-21.1 comes with ssl-9.0.2 and otp-21.1.1 comes with ssl-9.0.3. If you build a application release depending on ssl with erlang-21.1 and run it with erlang-21.1.1, it will look for ssl-9.0.2 (because of the application's .rel file and fail because only ssl-9.0.3 is available.
Sounds like what you would like is an erlang:21.1.0 image so that you can pin the version and not get automatic upgrades of the base erlang version.
The thing is that when you build your release you lock the versions of all applications into the boot file of the release. So when you later start that release those specific application version have to be available. rebar3 and relx solves this problem for you so that you don't have to worry about it.
@c0b: We do follow semver (or a variant very close anyways), you can read more about it here: http://erlang.org/doc/system_principles/versions.html.
I've tried to run an application depending on ssl build with Erlang 21.1 in a container based on erlang:21.1 and failed.
Would you share some of the failure log?
it will look for ssl-9.0.2 (because of the application's .rel file and fail because only ssl-9.0.3 is available.
The thing is that when you build your release you lock the versions of all applications into the boot file of the release. So when you later start that release those specific application version have to be available
is there some way to declare dependency like in Semantic Versioning, want ^ssl-9.0.2 to allow any ssl-9.0.2+ patch version upgrade?
Sorry folks for this late answer.
Sounds like what you would like is an erlang:21.1.0 image so that you can pin the version and not get automatic upgrades of the base erlang version.
Yes that's my point. More precisely I work with Elixir and a tool called Distillery to produce Erlang releases. Thus I can produce an Elixir application release that I can run it in a bare Erlang context. I don't see how rebar3 and relx could help me to somewhat "relocate" my release, and anyway I don't see how they fit with Distillery.
Would you share some of the failure log?
The only log I have is this (from my CI)
Could not locate code path for ssl-9.0.2!
and I think it is output from some boot script built by Distillery.
More precisely I work with Elixir and a tool called Distillery to produce Erlang releases. Thus I can produce an Elixir application release that I can run it in a bare Erlang context.
Wouldn't it just be better to run it in a bare debian/alpine context? Like what is done here: https://github.com/erlang/docker-erlang-example
Then you won't have any problems with the Erlang versions as everything Erlang+Elixir is packed into the applicatoion release. Distillery should be able to create such releases.
Indeed, that's what we've done so far: building a release with an embedded erts. But we wanted to be able to deploy not only bundling the release in a docker image, but just dropping it on a machine/VM with an already installed erlang runtime as well.
Then we discovered on the Elixir forum (https://elixirforum.com/t/erlang-releases-and-otp-versions/17771) the compatibility rules about applications and the erlang runtime versions.
So now my only point is whether to use the official erlang docker images, but we have this tagging issue, or producing our own images with with tags matching exactly the runtime versions.
is there some way to declare dependency like in Semantic Versioning, want ^ssl-9.0.2 to allow any ssl-9.0.2+ patch version upgrade?
so is there no way for an application to declare a loose dependency like ^ssl-9.0.2 to allow running with any future ssl-9.0.x with (x>=2) versions (but not ssl-9.1.x above)?
so is there no way for an application to declare a loose dependency like ^ssl-9.0.2 to allow running with any future ssl-9.0.x with (x>=2) versions (but not ssl-9.1.x above)?
Not when you make a release of your application. When you create the release, you pin the versions of your dependencies. There is nothing that says that you have to make a release however, but most projects tend to do that.
I think that it makes sense to have a 21.1.0 image. Since there exists 21.1.1 images that are immutable, it makes sense that there should be a 21.1.0.
If you want to cut down on the amount of images that created, I don't really see the point of the 21.x images. Either you want a specific X.Y.Z version, or you want to have release X. Though maybe there has been some discussion before about it that I have missed.
Hello folks, is there any update here? Any chance to have immutable images?
if this OTP-21.1.0 tag image is only a one-time request, we may manually add a 21.1 series into https://github.com/docker-library/official-images/blob/master/library/erlang
But, if this is asking to create 21.X.0 tags for all future new minor OTP-21.X releases, that would harder, and a question followed would be do you want 21.X.0.0 as well? when a major release number 21 is no longer current, there would be some 4 numbers releases
Or if upstream like to make future releases tags like when next 21.3 comes out, would there be a OTP-21.3.0 github tag? if so, it would be automatically supported by current scripts; but I don't think it's likely for upstream to make changes (the .0 tags)
So far I see it's like a one-time task? because only this OTP-21.1.1 was created incompatible with OTP-21.1, where applications depend on the builtin ssl lib of OTP-21.1
Well, as @garazdawi stated, I would like immutable images: whatever the tag it will not be used for anything else than the image it was initially used for. If 21.X is used to tag an OTP-21.X version, it will not eventually slide to an OTP-21.X.Y.
That said, if there is any trouble here, you can just close this issue and keep things as they are. Until now I used to produce my own Erlang images with the kind of immutability of tags I'm seeking. I just though it was a best practice to use something more "official" and share with the community.
So don't worry and keep the good work, I have a plan B 😄
if this OTP-21.1.0 tag image is only a one-time request, we may manually add a 21.1 series into https://github.com/docker-library/official-images/blob/master/library/erlang
I think that this overview page is fine the way it is. We don't want to expose nor encourage people to not take the latest patched version of Erlang by putting up 21.1 now that 21.2 has been released.
But, if this is asking to create 21.X.0 tags for all future new minor OTP-21.X releases
Yes, that is what we are discussing. Basically what would need to be done is that whenever a OTP-X.Y release is done, you would treat it as if a OTP-X.Y.0 release has been done. We (OTP-team) decided that we would not call our major/minor releases that for reasons that I can't remember right now, but they all have an implicit .0 added at the end. i.e.
OTP-21.0 -> OTP-21.0.0
OTP-21.0.1 -> OTP-21.0.1
OTP-21.1 -> OTP-21.1.0
We will never release a tag called OTP-21.0.0, we may however release a tag called OTP-21.0.0.1, but that would be very very rare.
a question followed would be do you want 21.X.0.0 as well? when a major release number 21 is no longer current, there would be some 4 numbers releases
no
Or if upstream like to make future releases tags like when next 21.3 comes out, would there be a OTP-21.3.0 github tag? if so, it would be automatically supported by current scripts; but I don't think it's likely for upstream to make changes (the .0 tags)
I don't think we can change the way that we tag releases. We break section 2 of https://semver.org/#spec-item-2 and I think that we will continue to do so.
@fcabestre easiest way to get around this is to include erts in your release ;)
There is a lot in this thread so I may have missed where a decision is made but I'll just throw in how I'd prefer it to work:
Tags are mutable for any part of the version they do not specify, like OTP-22, OTP-21.1, etc would be the latest 22 and 21.1 (today, 22.0.2 and 21.1.4).
OTP-21.1.0 would be just that, verison 21.1.0.
The only question is what to do about ones like for example OTP-21.3.7 which would be version 21.3.7.1.