Class attributes identical to inherited ones are wrongly deleted

[Daniel Fremont]

In 0.3.7, attributes in the __dict__ of a class are now deleted by dill if they have the same name and value as an attribute of a superclass. For example:

import dill

class Foo:
    x = 1

class Bar(Foo):
    x = 1  # identical value as superclass

assert 'x' in Bar.__dict__  # passes

data = dill.dumps(Bar)
Bar2 = dill.loads(data)

assert 'x' in Bar2.__dict__  # fails with dill 0.3.7 (works with 0.3.6)

This behavior is caused by the _get_typedict_type function added in #577. Based on its docstring it seems the idea was to exclude inherited methods, but the code actually excludes any attribute whose value is identical to the inherited value. This is not safe (arguably even in the case of methods) because as the code above demonstrates, it changes the behavior of code which inspects the __dict__ directly. Obviously the above code is a toy example, but we have real code which does this (basically to set a non-inherited flag on classes) and which was broken by dill 0.3.7.

In case it matters, I'm using Python 3.11.1 on macOS; we've also seen the problem under Python 3.10.12 on Ubuntu.

[Daniel Fremont]

Another way this issue can appear is that the __module__ of a subclass can be restored incorrectly when the class is pickled by value:

File test2.py:

class Foo:
    pass

class Bar(Foo):
    pass

BarAlias = Bar

del Bar  # to prevent pickling by reference

File test.py:

import dill

from test2 import BarAlias as Bar

print(Bar.__module__)  # prints `test2`

data = dill.dumps(Bar)
Bar2 = dill.loads(data)

print(Bar2.__module__)  # prints `dill._dill` under 0.3.7

The underlying problem is the same: the __module__ attribute of Bar doesn't get saved when pickling its __dict__, because the value of the attribute is identical to the value inherited from Foo. Then __module__ isn't included in the dict when creating the new class during unpickling, and so it gets the usual value based on where the class was made (inside dill._dill).

[Mike McKerns]

yeah, it looks like if value is base_value needs to exclude singletons/flyweights.

[Daniel Fremont]

Thanks very much for the quick fix, Mike!