Include rebar3 in alpine image
tsloughter opened this issue · comments
I think if the main image includes rebar3 it should be included with the alpine image as well. Images are usually used for development/building a release to use in another image anyway and thus the user has to manually get rebar3. Plus rebar3 is small in comparison.
from erlang:20: the building from source code generated a 3.28MB layer;
$ docker image history erlang:20
IMAGE CREATED CREATED BY SIZE COMMENT
2ce2f7f47d0c 4 days ago /bin/sh -c set -xe && REBAR3_DOWNLOAD_URL... 3.28MB
<missing> 4 days ago /bin/sh -c #(nop) ENV REBAR3_VERSION=3.5.2 0B
<missing> 4 days ago /bin/sh -c set -xe && REBAR_DOWNLOAD_URL=... 209kB
<missing> 4 days ago /bin/sh -c #(nop) ENV REBAR_VERSION=2.6.4 0B
<missing> 4 days ago /bin/sh -c #(nop) CMD ["erl"] 0B
<missing> 4 days ago /bin/sh -c set -xe && OTP_DOWNLOAD_URL="h... 237MB
<missing> 4 days ago /bin/sh -c #(nop) ENV OTP_VERSION=20.3.6 0B
<missing> 9 days ago /bin/sh -c set -ex; apt-get update; apt-... 556MB while the precompiled https://github.com/erlang/rebar3/releases/download/3.5.3/rebar3 file is 687KB, if use ADD to download it and it needs a chmod that duplicates the 704KB layer,
$ docker image history 37b80a602cc4
IMAGE CREATED CREATED BY SIZE COMMENT
37b80a602cc4 9 minutes ago /bin/sh -c chmod +x /usr/local/bin/rebar3 704kB
d8e6fca9565f About an hour ago /bin/sh -c #(nop) ADD 1b76493239bad949b5fc... 704kB Or use a similar apk add curl to download and chmod in a single layer approach, added a 852KB layer,
$ docker image history 38741a1ebba4
IMAGE CREATED CREATED BY SIZE COMMENT
38741a1ebba4 6 minutes ago /bin/sh -c apk add --no-cache --virtual .f... 852kB
9a723a5dfb19 4 days ago /bin/sh -c #(nop) CMD ["erl"] 0B I agree the rebar3 is the current dominating package manager, can be in the alpine image,
I'd like to see comments from more users and see which is the best approach bring in rebar3
I believe the pre-compiled option is better. As far as I know, the latest Alpine image(s) doesn't have cURL installed by default, but wget can be used and do what you said: use RUN with wget and chmod all in one command to avoid duplicating that layer.
PS: I did a quick search (in different branches) for the rebar3 package in Alpine and it's not there. So I guess that's not an option for now.
Preferably we would also install it and not use the escript. In my dockerfiles I do:
FROM erlang:20.3.6-alpine as builder
RUN apk add --update tar curl git bash make libc-dev gcc g++ vim && \
rm -rf /var/cache/apk/*
RUN set -xe \
&& curl -fSL -o rebar3 "https://s3.amazonaws.com/rebar3-nightly/rebar3" \
&& chmod +x ./rebar3 \
&& ./rebar3 local install \
&& rm ./rebar3
ENV PATH "$PATH:/root/.cache/rebar3/bin"The unpacked version is slightly faster to run and gets rid of some bugs that escripts have with running a shell.
Oh but the make, libc, gcc, g++ stuff doesn't need to be there, just curl I think. I just had those since I usually have some dep with a NIF and use tar when unpacking a release I build in the dockerfile.
So #135 is an example of an issue I have with docker image versioning in general.
Maybe there should be a "image version" after the OTP version? Like the first 20.3.6 that is published is 20.3.6-1, and then if it is upgraded with a new rebar3 it would bump to 20.3.6-2.
Or put the rebar3 version in the tag as well? But that makes for long, possibly confusing, tags like 20.3.6-3.5.3.
Any progress on this? It would be great to have Rebar 3 by default when using these Docker images to compile and test Erlang software
Or maybe it could be the other way around. The rebar3 project publishes the images with the latest compatible major version of Erlang that it's known works for it. In that sense, you can have multiple rebar3 images for multiple major versions of Erlang – until the point they can get retired/merged.
Gradle/Maven (build tools) does that for the Java Platform.
@x80486 Rebar 3 is owned and managed under the official erlang organization though, so it wouldn't be out of place to include it by default: https://github.com/erlang/rebar3
But how is that related with my previous comment? It looks to me that you don't want rebar3 to be bundled in the Erlang images...
Rebar 3 is owned and managed under the official erlang organization though, so it would be out of place to include it by default
...but you were indeed asking for it.
What I was trying to say is: if I want rebar3 because I'm planning to use to build an Erlang project, I don't care (much) about the Erlang version as long as I know I'm on the major version that I want. You could do images like (https://hub.docker.com/_/rebar3/):
3.9.1-otp21.1.1,3.9.1-otp21.1.1-alpine3.5.3-otp20.1.1,3.5.3-otp20.1.1-alpine
The build tool has the dependency over the programming language, not the other way around. Like, hypothetically, you wouldn't pull the Erlang image to have openssl-dev (provided it's kept in the image), but the openssl-dev image (assuming something like that exists).
Does docker have any sort of templating yet? So we could do a Dockerfile like:
FROM $OTP-IMAGE
... install rebar3 ...
And then easily build and publish an image that includes rebar3 for any OTP version we have an image of.
@x80486 Misspelling, I meant "wouldn't" 🙂
I know this is a slippery slope... but it also occurred to me that I'd still need a separate image in order to do CI because many projects still have git dependencies, esp during development.
Same is true for necessary libs for building NIFs.
But I think having an image that is ready to go for building 99% of erlang projects would be good and would require something like:
RUN apk add --no-cache --update tar git bash make libc-dev gcc g++ vim
This wouldn't be an issue if CircleCI and similar had like a 'cache image' ability, so I didn't have to build and publish these images manually for every new version, but could tell CircleCI to use image X with additions Y, then if I change x it rebuilds... But I don't think anything like that exists.
Ok, looks like CircleCI does indeed have something like this as a premium feature https://circleci.com/docs/2.0/docker-layer-caching/
@tsloughter Maybe you could export the image and abuse the normal caching in CircleCI? https://circleci.com/docs/2.0/caching/
both of the latest comments from docker-hub admins make sense, I think now it's better to just include rebar3 in the alpine images? how do other contributors here think
#171 is merged, and docker-library/official-images#4995 is made, not sure this will be merged into the official images.
The 21 and 20 have included rebar3, and they can be found in the docker hub now, close this issue.