Library not working when spark job deployed in databricks 7.3 LTS
DCCK opened this issue · comments
Library not working when spark job deployed in databricks 7.3 LTS with Spark 3.0.1 (Scala -2.12) + (net.heartsavior.spark:spark-sql-kafka-offset-committer_2.12:0.3.0-spark-3.0).
Following warning is displayed in the logs, however same is working in Local spark standalone cluster(with Apache Spark Distribution)
WARN KafkaOffsetCommitterListener: Cannot find query f835de4b-2919-4f60-b849-6082d294e4f7 from active spark session!
Is this library works fine with Kafka SASL SSL?
Please let me know what is the issue.
The project is not intended to support vendors' distribution. You may need to fork the project and fix it by yourself, or contact Databricks.
Please setup the same for Apache distribution of Kafka and Spark, and test it again to report it. Thanks for the interest.