uqfoundation / dill

serialize all of Python

Home Page:http://dill.rtfd.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

tqdm is not serializable?

mishadr opened this issue · comments

dill 0.3.7
tqdm 4.66.1
When I try to serialize tqdm object:

from dill import dumps, loads
from tqdm import tqdm

b = dumps(tqdm())
print(loads(b))

I get a RecursionError:

0it [00:00, ?it/s]
Traceback (most recent call last):
  File "/home/misha/workspace/gnn_interpretation/src/experiments/tqdm_bug.py", line 5, in <module>
    print(loads(b))
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/dill/_dill.py", line 301, in loads
    return load(file, ignore, **kwds)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/dill/_dill.py", line 287, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/dill/_dill.py", line 442, in load
    obj = StockUnpickler.load(self)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/utils.py", line 139, in __getattr__
    return getattr(self._wrapped, name)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/utils.py", line 139, in __getattr__
    return getattr(self._wrapped, name)
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/utils.py", line 139, in __getattr__
    return getattr(self._wrapped, name)
  [Previous line repeated 992 more times]
RecursionError: maximum recursion depth exceeded
Exception ignored in: <function tqdm.__del__ at 0x7ff5ade079d0>
Traceback (most recent call last):
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/std.py", line 1149, in __del__
    self.close()
  File "/home/misha/workspace/gnn_interpretation/venv/lib/python3.8/site-packages/tqdm/std.py", line 1268, in close
    if self.disable:
AttributeError: 'tqdm' object has no attribute 'disable'

This could be connected to AttributeError: 'tqdm' object has no attribute '' #487 , but the issue seems to be solved.

It would seem there's a recursive reference pointer of some kind. You can, at first blush, try increasing the max recursion depth, or you can try using dill.settings['recurse'] = True (which changes the recursion behavior in globals during serialization).

Thanks for the reply, but that doesn't help..

Ok, if that's the case, thentdqm.tdqm is not serializable.

Probably the most appropriate thing to do is for the tqdm class to add a __reduce__ method or some other state-capturing method for serialization. I'm closing this, but feel free to reopen if you feel your question hasn't been answered.