Distributed processing [Question]
patchthecode opened this issue · comments
Since the cost of this awesome compression tool is processing/ram,
Is there any way the processing can be done on a distributed system (multiple systems doing the compression) ?
Unfortunately after every bit of input, all the data needs to synchronize in order to advance. Synchronizing the data between different systems after every bit has a lot of overhead, so it would make things really slow. This is also the reason that cmix uses only a single thread - multithreading slows things down for the same reason.
thanks.