qubole / sparklens

Qubole Sparklens tool for performance tuning Apache Spark

Home Page:http://sparklens.qubole.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Not able to run spark lens on spark history file

Gaurang033 opened this issue · comments

hi Guys,

I am trying to get spark lens report from my history file. However it's not able to identify the arguments.

I am getting following error

[sa_awsprd_nztoaws@ip-10-160-1-109 gshah]$ spark-submit --jars sparklens-0.3.2-s_2.11.jar  --class com.qubole.sparklens.app.ReporterApp qubole-dummy-arg  file:////home/gshah/application_1589573238486_1924 source=file
20/09/03 14:57:22 WARN DependencyUtils: Local jar /home/gshah/qubole-dummy-arg does not exist, skipping.
#
# java.lang.OutOfMemoryError: Java heap space
# -XX:OnOutOfMemoryError="kill -9 %p"
#   Executing /bin/sh -c "kill -9 3788"...
Killed

@Gaurang033 looks like you are running our of driver heap. Try increasing driver memory:
spark-submit --driver-memory 4g --jars sparklens-0.3.2-s_2.11.jar --class com.qubole.sparklens.app.ReporterApp qubole-dummy-arg file:////home/gshah/application_1589573238486_1924 source=file