Stonesjtu / pytorch_memlab

Profiling and inspecting memory in pytorch

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

weakly-referenced object no longer exists

vsantjr opened this issue · comments

Hello,

Firstly, congratulations for memlab. I have been trying to use it in Google Colab, but sometimes this error happens:

ReferenceError Traceback (most recent call last)
in ()
33 print('Reporter!!!!!!!')
34 reporter = MemReporter()
---> 35 reporter.report()

2 frames
/usr/local/lib/python3.7/dist-packages/pytorch_memlab/mem_reporter.py in (.0)
62 #FIXME: make the grad tensor collected by gc
63 objects = gc.get_objects()
---> 64 tensors = [obj for obj in objects if isinstance(obj, torch.Tensor)]
65 for t in tensors:
66 self.device_mapping[t.device].append(t)

ReferenceError: weakly-referenced object no longer exists

In my code, I use MemReport() just right after the training phase, i.e.:

for epoch in range(num_epochs):
net.train()
...
# end of training

print('Reporter!!!!!!!')
reporter = MemReporter()
reporter.report()

Do you know what is the problem?

Thank you and regards.

This error message is weird.

Could you plz post the:

  • memlab version
  • pytorch version
  • the computation devices (GPU type / CPU)

btw could you try adding a torch.cuda.synchronize before creating a reporter.

Hello,

I am using Google Colab. I just ran !pip install pytorch_memlab in the notebook. The PyTorch version is the one of Colab. The GPU I guess that is K80 but I am not so sure.

Thank you.