Negative 'diff max' and 'diff peak' values?

[ProGamerGov]

What do the negative values mean? For example below is part of what I captured when running some code:

Line # Max usage   Peak usage diff max diff peak  Line Contents
===============================================================
    56                                           @profile
    58                                           def main():
    59     0.00B        0.00B   -4.14G   -4.64G      dtype, multidevice, backward_device = setup_gpu()

[ProGamerGov]

I just realized that they mean that the difference went from higher memory usage to lower memory usage.

[Kaiyu Shi]

The prefix diff means the memory change from the previous line to the current line.

The negative value means the memory usage is decreasing, which happens when you free some GPU tensors (or python's built-in GC frees the GPU tensors).

[Kaiyu Shi]

It's a common case when you switch the target_gpu to profile on.