qubole / sparklens

Qubole Sparklens tool for performance tuning Apache Spark

Home Page:http://sparklens.qubole.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

request: option to add spark.job.description to the report

Pooj2 opened this issue · comments

commented

While reading the for example the Printing Application timeline, looking at 535 jobs it is hard to relate a number to an action in the Spark program. In the Spark UI there is a solution for this: set the spark.job.description and/or spark.jobGroup.id in the driver for every (eager) action you want the Spark executors to do (see the answers to https://stackoverflow.com/questions/39123314/how-to-add-custom-description-to-spark-job-for-displaying-in-spark-web-ui)

Is it possible to add an option to Sparklens to choose to report the spark.job.description and/or spark.jobGroup.id with its Job id?