would tensorflow profiler consider profiling per-step system-level metrics?

Question

would tensorflow profiler consider profiling per-step system-level metrics?

burgerkingeater opened this issue 4 years ago · comments

Hi there,
As system-level metrics(CPU utilization, GPU utilization, memory...) are important indicators to debug training performance, would tensorflow profiler consider adding them? E.g. visualize how each step's CPU utilization varies.

ckluk · Answer 1 · Fri Jul 31 2020 02:43:31 GMT+0800 (China Standard Time)

Hi, Thanks for your suggestion. I have filed your request to an internal tracker and will consider its feasibility.

…

-ck

On Wed, Jul 29, 2020 at 9:01 PM Cheng Ren ***@***.***> wrote: Hi there, As system-level metrics(CPU utilization, GPU utilization, memory...) are important indicators to debug training performance, would tensorflow profiler consider adding them? E.g. visualize how each step's CPU utilization varies. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#115>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE33L3KWOAJO5QUNZVNI6NDR6DWAVANCNFSM4PMXPDOA> .

Cheng Ren · Answer 2 · Fri Jul 31 2020 02:49:25 GMT+0800 (China Standard Time)

@ckluk thanks, we are interested in this usecase and plan to make tensorflow core produce these metrics, and have profiler be able to visualize them. Would you be interested in reviewing the PR if it's done?