Record class needs to be defined in a module in distributed mode
GreatYYX opened this issue · comments
GreatYYX commented
This issue is caused by incomplete serialization of Record class defined in __main__
.
GreatYYX commented
Two possible solutions here:
- let upstream provide the feature to pickle interactively defined property type and the instance of the type.
- use another way to indicate if the property should be cached: using a function decorator to update indicators in
__dict__
instead of using self-defined type of property.
GreatYYX commented
Upstream provided another solution: override __reduce__
in cached_property
:
def __reduce__(self):
return cached_property.__new__, (cached_property, ), {'func': self.func}