Spark with Java 11

Question

Spark with Java 11

apssouza22 opened this issue 3 years ago · comments

I am using the image bde2020/spark-worker:3.0.0-hadoop3.2-java11 which I was expecting to contain Java 11 but it has the Java 8. Any away to run a Java 11 program with this image?

Gezim Sejdiu · Answer 1 · Thu Feb 04 2021 07:24:33 GMT+0800 (China Standard Time)

Hey @apssouza22 ,

thanks a lot for using our docker images.

Yes, that is true. The changes contributed here #104 have been merged to that branch and the images will be deployed on Docker Hub soon.

Please, do let me know if you still do not get the Java 11 so that we can fix it.

Best regards,

Rob Vadai · Answer 2 · Sat Feb 06 2021 04:47:35 GMT+0800 (China Standard Time)

Can you advise when the java11 image will have Java 11 installed in it.

Thanks

Rob Vadai · Answer 3 · Sat Feb 06 2021 06:24:06 GMT+0800 (China Standard Time)

If you look closely on the java11 branch, Dockerhub shows a build failure - is it the case why the image contains Java 8?

Gezim Sejdiu · Answer 4 · Sat Feb 20 2021 06:06:27 GMT+0800 (China Standard Time)

Hey @robvadai ,

you were right about Java 11 :( -- we missed it while using the base docker image (not specifying to use with Java 11) :( -- I just did update it to use Java 11 instead.

The docker-images are being buld on Docker Hub (it may take a bit till they are available there), but I do confirm that it is picking up the java 11 (tested locally via docker-compose up):

Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 3.0.0
      /_/
         
Using Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 11.0.4)

Once again, thank you for pointing this issue out.

PS: I'm closing this issue for now, in case it still persists, please, feel free to comment so that we can solve it.

Best regards,
Gezim