scala/math/Ordering not found after 2.5.0 update

[Ximo Guanter]

I am using embedded-kafka 2.4.1.1 and it works like a charm. I have tried to update the dependency to 2.5.0, 2.5.1 or 2.6.0 but I get this stacktrace with the newer versions:

[info]   Cause: java.lang.ClassNotFoundException: scala.math.Ordering$$anon$7
[info]   at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
[info]   at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
[info]   at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
[info]   at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
[info]   at kafka.api.ApiVersion$.orderingByVersion(ApiVersion.scala:45)
[info]   at kafka.api.ApiVersion.compare(ApiVersion.scala:141)
[info]   at kafka.api.ApiVersion.compare$(ApiVersion.scala:140)
[info]   at kafka.api.KAFKA_2_6_IV0$.compare(ApiVersion.scala:348)
[info]   at kafka.api.KAFKA_2_6_IV0$.compare(ApiVersion.scala:348)
[info]   at scala.math.Ordered.$greater$eq(Ordered.scala:91)

I am using Scala 2.12.12

[Francesco Pellegrini]

Hi @edrevo,
did you try applying this change? I see that Dependencies.scala still refers to 2.12.11 version, unfortunately...

[Mårten Rånge]

I recently ran into this myself when we went from scala 2.12.10 to scala 2.12.12.

From a bit of digging and chatting to the scala maintainers (scala/bug#12162) my belief this is because kafka core inlines scala library hidden class names for performance reasons: https://github.com/apache/kafka/blob/trunk/build.gradle#L520

In kafka they know this leads to ABI compatibility issues but they say:

Inline more aggressively when compiling the core jar since it's not meant to be used as a library.

It seems though EmbeddedKafka is using core as a library (ApiVersion.scala is part of kafka core structure) for its function and thus the inlining of scala library hidden names causes headaches.

One way is locking the scala library dependency to 2.12.10 for our test suites but it's possible other dependencies will fail if they don't get 2.12.12 as they expect.

I have no good idea on how to go on about fixing this as it seems kafka core intentionally is compiled to make minor versions upgrades possibly not work (for performance reasons) as it's not intended to be used a library.

Shadowjar:ing the kafka dependency could maybe work but as it's the core scala library that needs to be shaded it makes it tricky.

Anyway, just wanted to add a bit more context to this issue.

PS. About scala inlining: https://www.lightbend.com/blog/scala-inliner-optimizer

[Enno Runne]

As Kafka core inlines the Scala library, you can not use a different Scala patch version than what Kafka used to compile its jars. Embedded Kafka for Kafka 2.6.0 requires users to be on Scala 2.12.11 or 2.13.2.

akka/alpakka-kafka#1212 (comment)

[Francesco Pellegrini]

Kafka 2.7.0 bumps Scala dependencies to 2.12.12 and 2.13.3 respectively.