Giters

miurahr / pyppmd

pyppmd provides classes and functions for compressing and decompressing text data, using PPM (Prediction by partial matching) compression algorithm variation H and I.2. It provide an API similar to Python's zlib/bz2/lzma modules.

Home Page:https://pyppmd.readthedocs.io/en/latest/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

consumer-producer problem

miurahr opened this issue · comments

commented

Describe the bug

Decompressor become wrong status when memory_size is smaller than file size.

Related issue
#25

To Reproduce

The test has already merged into main branch.

Expected behavior

PyPPMd should handle it correctly.

Environment:

  • OS: no dependency
  • Python: all python versions
  • project version: all released versions, main branch.

Additional context

The root cause is the producer-consumer problem. Now producer use Python buffer protocol that provide a buffer, but consumer expect it can read arbitrary data from buffer.
The solution is to run the library code,consumer, in another thread and stop and wait when input buffer is exhausted.
When returns to caller code by need-input-data status, it go back to user python code and wait a next call, and when input is given, it wake the library code to continue.