Use a simple dictionary-based compression on the filenames
ShaheedHaque opened this issue · comments
Thanks for this library. My interest in it is in sending exceptions across a Celery-to-parent multiprocess/billard process boundary, and also in storing the result in the Celery result backend. Some of this is moderately expensive/limited and it would be very useful to reduce the data involved. Given that a typical to_dict() representation is dominated by the filenames:
'tb_frame': {
'f_globals': {
'__file__': '/main/srhaque/redacted/testdjango/redacted/tasks.py',
'__name__': '?'
},
'f_code': {
'co_filename': '/main/srhaque/redacted/testdjango/redacted/tasks.py',
'co_name': 'function_run'
}
},
'tb_lineno': 601,
'tb_next': {
I implemented a simple dict-based "compression" scheme that eliminates the redundant filenames. Would an API-compatible patch to to_dict()/from_dict() for this purpose be of interest? If so, I'd be happy to provide a PR.
So my concern is that if we have this in tblib it would draw people to make it more and more efficient (and more complicated at the same time). I would rather have this sort of code as a documentation/cookbook example, provided it's not very big.
I see. I'm thinking the change is not very big, but it is intrusive. Perhaps I should make a PR, and then you can see what you think either about taking it, or not...at least its available if needed.