Step time in Overview and in Trace don't match

[David Grigorian]

Hi all!
Trying to use torch.profiler to measure acceleration in inference of my model
https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html
I am looking at the profiling results in Tensorboard and the following question arises:
Why are the step time in the overview and the step time in the trace not the same?
Is it a bug in the torch plugin for Tensorboard?

[Aaron Shi]

There may be a bug causing a difference between the Tensorboard plugin and trace annotation. Could you please provide a screenshot of what you are seeing in both places?

[zackerzzz]

i printed the step time in code, is about 30ms, but the overview shows 200ms.

import time
prof = torch.profiler.profile(
        schedule=torch.profiler.schedule(wait=1, warmup=1, active=5, repeat=2),
        on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/resnet18'),
        # record_shapes=True,
        profile_memory=True,
        with_stack=False)
prof.start()
for step, batch_data in enumerate(train_loader):
    st = time.time()
    print("step:{}".format(step))
    if step >= (1 + 1 + 5) * 2:
        break
    train(batch_data)
    print((time.time()-st)*1000 ,"ms")
    prof.step()
prof.stop()