Can't install rabbit_common on Elixir

[Michael Schaefermeyer]

Hey,

Thanks for the official images, super excited about them, especially the alpine variants! I like that you keep the alpine image as small as possible, as it's normally trivial to install additional dependencies.

Unfortunately there is an issue installing rabbit_common, which is caused by OTP20 no longer including some libraries. It's described here:
rabbitmq/rabbitmq-common#27

The suggested fix - installing erlang-src fails, as it is not a package that's available from alpine. Maybe @msaraiva can help here, too.

I've illustrated the issue in a small project:
https://github.com/invrs/rabbit_common_problem/blob/master/Dockerfile

Thanks,
Michael

[c0b]

1. the alpine image building process does delete ./src from each lib but does not delete ./include for depended code compilation; if you browse file inside the container, a lot of header files like /usr/local/lib/erlang/lib//include/.hrl do exist in the alpine image as well
2. the ssl lib looks like a different one, because a good convention for erlang lib design is: if the header is shared for other code to compile, the header file should be put under its ./include directory, but headers file under ./src is more of this library's internal use only;
   apparently, this header file "ssl/src/ssl_api.hrl" is breaking the convention,

however, I checked all the recent erlang versions, it seems the "ssl/src/ssl_api.hrl" is always there under ./src and got included by third-party libraries (the rabbitmq-common); it's not because of OTP20 no longer include this library

1. https://github.com/erlang/otp/tree/maint-20/lib/ssl/src
2. https://github.com/erlang/otp/tree/maint-19/lib/ssl/src
3. https://github.com/erlang/otp/tree/maint-18/lib/ssl/src
4. https://github.com/erlang/otp/tree/maint-17/lib/ssl/src

an ideal solution may be: send a PR to https://github.com/erlang/otp to fix this header file location, move from ./src to ./include for 3rd party code, then fix rabbitmq-common to switch to include header file by ssl/include/ssl_api.hrl

this approach I think is ideal, to follow above design convention, but it may take long time for all "ssl/src/ssl_api.hrl" users to fix

a short term fix in your Dockerfile could be: use multi-stage build, copy the ssl/src/ from erlang:20 or elixir:1.5 to the stage of your build

[Michael Schaefermeyer]

Hey @c0b,

If I understand that thread in rabbit-common correctly they omitted those header files from main on purpose. They are now included in the erlang-src package and no longer in the main file.
I doubt that submitting a PR for something they seem to have done intentionally is going to help.

We could either compile erlang with erlang-src included, or we could make sure that we can install erlang-src manually.

FWIW the multi-stage build works but it's a big workaround.

[c0b]

as I am researching how many packages from the main erlang-otp have similar break-convention public headers file in src instead of include; it looks like a lot!
https://github.com/erlang/otp/tree/maint-20/lib/

so my concern is even move one header file to better location ssl/include/ssl_api.hrl may not be enough

solution to include erlang-src sure easy, just delete src from the rm line; will result somewhat larger image size for sure;
another one solution could be remove src/*.erl for each lib instead of remove its whole src, or say make someway to keep src/*.hrl headers during image building. will research

[Michael Schaefermeyer]

Thanks, I appreciate the insights and diligence.

Let me know if I can be of help.

[c0b]

I've done some local build test, shows that if leave all erlang/lib/*/src untouched, will result an 108MB image, if remove all erlang/lib/*/src except src/*/*.hrl header files, result 67.7MB; only 2 MB increase from current no src 20.0-alpine image; have a PR follows

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
otp                 20.0-alpine-3       1f30760c6669        4 minutes ago       67.7MB
otp                 20.0-alpine-2       f032f2b8792e        35 minutes ago      108MB
otp                 20.0-alpine         eecb9efa192e        4 days ago          65.7MB

[c0b]

just wait erlang/docker-erlang-otp#65 for erlang:alpine to go to official and then elixir:alpine
or you may test run this private c0b0/elixir:v1.5.1-alpine-2 image if you want; it's the private header files made the 2MB additional

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
c0b0/elixir         v1.5.1-alpine-2     f725db7d43aa        4 minutes ago       73.3MB
elixir              v1.5.1-alpine-2     f725db7d43aa        4 minutes ago       73.3MB
elixir              1.5.1-alpine        8d82b86445e5        2 weeks ago         71.3MB

[Michael Schaefermeyer]

Hey @c0b,

Thank you very much, that looks great! Will report back once the build works with this!

Thanks,
Michael

[c0b]

although mentioned resolve in erlang/docker-erlang-otp@67e6c5f the elixir images still wait a refresh.

[Michael Schaefermeyer]

Confirmed that with the current image everything works beautifully! Thanks so much!