joblib / joblib

Computing with Python functions.

Home Page:http://joblib.readthedocs.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Memory considers 1 different than 1.0 which can be unexpected

thiswillbeyourgithub opened this issue · comments

Hi,

I find myself in situation were I sometime call a function using floats or ints, depending on the preprocessing of some gradio components.

I noticed that calling a cached function using floats will not use the value from the cache and instead recompute it. For example calling with arg=1 will not reuse arg=1.0

I can workaround it using a wrapper function that turns int into floats but I think this is somewhat unexpected.

This also makes me think that for custom class that implement an eq method, caching can be completely unexpected because although they have the same value, they would be considered different by Memory

Reproduction code:

from joblib import Memory
m = Memory("", verbose=False)
@m.cache
def f(a):
     print("not cached")

f(1) # not cached
f(1)
f(1.0) # not cached
f(1.0)
1 is 1.0 # False
1 == 1.0 # True

Agreed. I think that the documentation should state somewhere the exact rules that are used to know if the cache needs to be recomputed.

I expected that the __eq__ method would be used in some cases.

Edit: for anyone wanting a decorator that replaces int by floats:

def floatizer(func):
    "used to cast the ints as float to make sure the cache is used"
    def wrapper(*args, **kwargs):
        args = [float(ar) if (isinstance(ar, int) and not isinstance(ar, bool)) else ar for ar in args]
        kwargs = {k: float(v) if (isinstance(v, int) and not isinstance(v, bool)) else v for k, v in kwargs.items()}
        return func(*args, **kwargs)
    return wrapper

use like so:

@floatizer
@the_cache.cache
def my_function(some, args, and, kwargs):
    pass