hortonworks-spark / shc

The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Hang connection issue when write on hbase

attolini opened this issue · comments

Hello everyone,
I'm experiencing hang connection issue when I try to write on Hbase table with a Spark Shell.
I launch the Spark Shell as indicates in README:
spark-shell --packages com.hortonworks:shc-core:1.1.1-2.1-s_2.11 --repositories https://repo.hortonworks.com/content/groups/public/ --files /etc/hadoop/conf/hbase-site.xml --jars /usr/hdp/current/phoenix-client/phoenix-server.jar

and I'm stucked with this:
scala> sc.parallelize(newData).toDF.write.options(Map(HBaseTableCatalog.tableCatalog -> cat, HBaseTableCatalog.newTable -> "5")).format("org.apache.spark.sql.execution.datasources.hbase").save()
[Stage 0:===================================================> (7 + 1) / 8]

Actually, if I interrupt the task and I run df.show, I see the new value in the table. So this is only an annoying hanging issue.
If I use /usr/hdp/current/phoenix-client/phoenix-client.jar I solve the issue. Any tips to solve this issue?
Best regards,
Alessio