Using Run Length Encoding in `sbo_preds` objects.
vgherard opened this issue · comments
For N
-gram models with N >= 3
, using Run Length Encoding for k-gram prefixes in sbo_preds
objects could bring two benefits:
- Reduce size of these objects.
- Make the retrieval of k-gram prefixes more efficient.