Failed to use a custom method, reporting an error that method does not exist in the JVM
weichi-ys opened this issue · comments
about information：
py4j version:0.10.9
python version:3.8
error information:
py4j.protocol.Py4JError: org.apache.kyuubi.engine.spark.SparkSQLEngine.getSparkSession does not exist in the JVM
@weichi-ys it should work I believe. Do you have a reproducer?
Looks like you're trying to access SparkSQLEngine companion object (https://github.com/apache/kyuubi/blob/8e8d8be8d2bd6b00570143602367573d56fb7825/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/kyuubi/engine/spark/SparkSQLEngine.scala#L152).
Scala creates a different class for JVM to use in this case. You should be able to access it via:
getattr(getattr(jvm.org.apache.kyuubi.engine.spark, "SparkSQLEngine$"), "MODULE$").getSparkSession()