ionelmc / python-tblib

Serialization library for Exceptions and Tracebacks.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Use a simple dictionary-based compression on the filenames

ShaheedHaque opened this issue · comments

Thanks for this library. My interest in it is in sending exceptions across a Celery-to-parent multiprocess/billard process boundary, and also in storing the result in the Celery result backend. Some of this is moderately expensive/limited and it would be very useful to reduce the data involved. Given that a typical to_dict() representation is dominated by the filenames:

 'tb_frame': {
    'f_globals': {
      '__file__': '/main/srhaque/redacted/testdjango/redacted/tasks.py',
      '__name__': '?'
    },
    'f_code': {
      'co_filename': '/main/srhaque/redacted/testdjango/redacted/tasks.py',
      'co_name': 'function_run'
    }
  },
  'tb_lineno': 601,
  'tb_next': {

I implemented a simple dict-based "compression" scheme that eliminates the redundant filenames. Would an API-compatible patch to to_dict()/from_dict() for this purpose be of interest? If so, I'd be happy to provide a PR.

So my concern is that if we have this in tblib it would draw people to make it more and more efficient (and more complicated at the same time). I would rather have this sort of code as a documentation/cookbook example, provided it's not very big.

I see. I'm thinking the change is not very big, but it is intrusive. Perhaps I should make a PR, and then you can see what you think either about taking it, or not...at least its available if needed.