qubole / sparklens

Qubole Sparklens tool for performance tuning Apache Spark

Home Page:http://sparklens.qubole.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Release for Scala 2.12

janekdb opened this issue · comments

Hi,

Any plans to release for Scala 2.12?

Scala 2.13 and Spark 3 would be good to have support for.

Thanks,
Janek

Just change Scala and Spark versions in build.sbt and rebuild the package with sbt compile && sbt package
Scala 2.12 and Spark 2.4.0 worked for me.

Does Qubole have any plans to release a build like that to spark-packages?

Hey, any update on this topic?

Scala 2.12 support would be nice. For myself, i'm using GCP DataProc with an image of Apache Spark 3.1.3 and Scala 2.12.14.
https://cloud.google.com/dataproc/docs/concepts/versioning/dataproc-release-2.0

Having sparklens as a supported listener solution in this environment with an emailed report would be really nice, because my organization tends to restrict FW rules aggressively that makes it very difficult to utilize many of the UI based diagnostic tools.

Just change Scala and Spark versions in build.sbt and rebuild the package with sbt compile && sbt package Scala 2.12 and Spark 2.4.0 worked for me.
@oleksiilopasov thanks, but this did not work for me. Compilation failed, SparkHadoopUtil is not longer accessible, along with some other errors.