qubole / sparklens

Qubole Sparklens tool for performance tuning Apache Spark

Home Page:http://sparklens.qubole.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

analysize spark eventhistory , but per stage metrics, max task mem is all 0.0KB

opened this issue · comments

sparklens-0.3.2-s_2.11.jar
spark version: 2.4.0
why the column maxTaskMem is 0.0KB, i try 2 different tasks, all shows like below:
image

I think they already have an answer here: #21 . Sounds like sparklens doesn't work with event logs mode properly for this case due to Spark's JsonProtocol class