Support for Spark 3.2.0+ with scala 2.13

Question

Support for Spark 3.2.0+ with scala 2.13

sheilahu opened this issue 2 years ago · comments

Sheila Hu commented 2 years ago

Hi,

I am wondering if there is plan to support spark with scala 2.13 and what the timeline looks like.

Thank you.

Jun Shi · Answer 1 · Fri Jun 03 2022 23:27:42 GMT+0800 (China Standard Time)

This is a PR for Spark 3.2 + 2.12. I plan to merge it this weekend.
Scala 2.13 is tricky, I will see how easy it is. We don't have a timeline for it. This project is user-driven: we are not actively developing it any more, but we rely on community's contribution for new features.

Jun Shi · Answer 2 · Sun Jun 05 2022 21:18:26 GMT+0800 (China Standard Time)

v0.5.0 (Spark 3.2 + Scala 2.13) has been published.
https://search.maven.org/artifact/com.linkedin.sparktfrecord/spark-tfrecord_2.13/0.5.0/jar
Please let me know if it works for you.