cwensel / cascading

Cascading is a feature rich API for defining and executing complex and fault tolerant data processing flows locally or on a cluster.

Home Page:http://cascading.wensel.net/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Possible to keep java8 on 4.5 release?

daniel-sudz opened this issue · comments

hey @cwensel any chance we could keep java8 on the 4.5 branch at least? AWS EMR 6.X (latest release) still defaults on java8 for the hadoop3 flavors and it would be much appreciated to keep this support before they drop it. I would imagine that EMR is a very common use case for this type of library.

Curious, is the jvm configurable on startup? Is it unstable on Java 11?

The Hadoop requirement is to compile Hadoop on Java 8, but it should theoretically run fine on Java 11. Unsurprised if that isn't true.

We can revert, but I really was hoping to start leveraging Java 11 without introducing yet another concurrent version.

it's possible to override but it's just not the default out of the box on AWS EMR and I would imagine there could be some breaking assumptions that would be a huge pain to deal with.

Java 8 is the default Java Virtual Machine (JVM) for cluster instances created using Amazon EMR release version 5.0.0 or later. 

https://docs.aws.amazon.com/emr/latest/ReleaseGuide/configuring-java8.html

I have have no doubt that hadoop itself works fine on java11. If it's a dealbreaker that's fine just would be more convenient in a managed runtime.

Fair enough, I'll revert the build version.

thank you @cwensel if this is pushed to say 4.6/4.7 I think that is reasonable. I would assume that newer release labels from AWS EMR will be java11+ within a year or so.