Kotlin / kotlin-spark-api

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

For the future: Remove KSparkSession using new context()

Jolanrensen opened this issue · comments

https://github.com/Kotlin/KEEP/blob/master/proposals/context-receivers.md

Context receivers will allow functions like

@context(SparkSession)
inline fun <reified T> List<T>.toDS() = toDS(spark)

to function without needing a separate KSparkSession. It could maybe also help in other places! It's still experimental though.

@asm0dey Pretty interesting idea indeed!
I got curious, so I removed KSparkSession using the new context receivers and while IntelliJ does not seem to agree, it does work!
You can have a look yourself and experiment if you're curious on my branch https://github.com/Jolanrensen/kotlin-spark-api/tree/1.6.20-M1

Edit: after a restart, IntelliJ agrees too XD What a fun new feature