Deque with JSONDisk throws TypeError: a bytes-like object is required, not 'int'
BubuOT opened this issue · comments
Marcus Hoffmann commented
I tried using a Deque like this:
Deque.fromcache(Cache(eviction_policy="none", disk=JSONDisk))
and while I'm not yet entirely sure if that's a useful thing (I guess some of the serialization caveats also apply to Deque's even if there's no key to serialize?) it probably shouldn't crash like this:
>>> deque = Deque.fromcache(Cache(eviction_policy="none", disk=JSONDisk))
>>> deque += range(200)
>>> deque[0]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/persistent.py", line 204, in __getitem__
return self._index(index, self._cache.__getitem__)
File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/persistent.py", line 161, in _index
for key in self._cache.iterkeys():
File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/core.py", line 2252, in iterkeys
yield _disk_get(key, raw)
File "/home/marcus/.pyenv/versions/3.10.7/lib/python3.10/site-packages/diskcache/core.py", line 360, in get
return json.loads(zlib.decompress(data).decode('utf-8'))
TypeError: a bytes-like object is required, not 'int'
Grant Jenks commented
That is a bug. Unfortunately, it doesn't have a great fix.
Deque uses the Cache's push() and pull() API which uses special cache keys that don't play nice with the key-serialization done in JSONDisk.
Here's the workaround:
In [6]: class JSONDiskValue(dc.JSONDisk):
...: put = dc.Disk.put
...: get = dc.Disk.get
In [7]: deque = dc.Deque.fromcache(dc.Cache(eviction_policy="none", disk=JSONDiskValue))
In [11]: deque += range(200)
In [12]: deque[0]
Out[12]: 0