grantjenks / python-diskcache

Python disk-backed cache (Django-compatible). Faster than Redis and Memcached. Pure-Python.

Home Page:http://www.grantjenks.com/docs/diskcache/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Cannot change JSONDisk compression level

schance995 opened this issue · comments

I tried to change JSONDisk's compression level, but it seems that setting disk parameters at cache initialization doesn't work. To test this, I wrote a script that reuses the JSONDisk example and prints the compression level:

from diskcache import Cache, Disk, UNKNOWN
import json
import zlib

class JSONDisk(Disk):
    """Cache key and value using JSON serialization with zlib compression."""

    def __init__(self, directory, compress_level=1, **kwargs):
        """Initialize JSON disk instance.

        Keys and values are compressed using the zlib library. The
        `compress_level` is an integer from 0 to 9 controlling the level of
        compression; 1 is fastest and produces the least compression, 9 is
        slowest and produces the most compression, and 0 is no compression.

        :param str directory: directory path
        :param int compress_level: zlib compression level (default 1)
        :param kwargs: super class arguments

        """
        self.compress_level = compress_level
        print(f'Init: {self.compress_level}')
        super().__init__(directory, **kwargs)

    def put(self, key):
        print(f'Put: {self.compress_level}')
        json_bytes = json.dumps(key).encode('utf-8')
        data = zlib.compress(json_bytes, self.compress_level)
        return super().put(data)

    def get(self, key, raw):
        print(f'Get: {self.compress_level}')
        data = super().get(key, raw)
        return json.loads(zlib.decompress(data).decode('utf-8'))

    def store(self, value, read, key=UNKNOWN):
        print(f'Store: {self.compress_level}')
        if not read:
            json_bytes = json.dumps(value).encode('utf-8')
            value = zlib.compress(json_bytes, self.compress_level)
        return super().store(value, read, key=key)

    def fetch(self, mode, filename, value, read):
        print(f'fetch: {self.compress_level}')
        data = super().fetch(mode, filename, value, read)
        if not read:
            data = json.loads(zlib.decompress(data).decode('utf-8'))
        return data


c1 = Cache(disk=JSONDisk)
c9 = Cache(disk=JSONDisk, disk_compress_level=9)

print('Storing in c1')
c1['1'] = 'one'
print('Storing in c9')
c1['9'] = 'nine'

print('Retrieving from c1')
c1['1']
print('Retrieving from c9')
c1['9']

The expected output is for c1 to print 1 and c9 to print 9. But instead c9 only prints 9 at initialization and prints 1 otherwise:

Init: 1
Init: 9
Storing in c1
Put: 1
Store: 1
Storing in c9
Put: 1
Store: 1
Retrieving from c1
Put: 1
fetch: 1
Retrieving from c9
Put: 1
fetch: 1

I tried this across python 3.11 and 3.12, and diskcache 5.6 all the way back to 4.0, and all pairs of python/diskcache versions failed to produce the expected result.

This would also affect all subclasses of Disk, since any disk_ arguments will be ignored.

You are using c1 everywhere. If I instead change the access to c9['9'] then I see the expected results