Index built incorrectly at chunking boundaries
newhoggy opened this issue · comments
The following gzipped Json file fails to deserialise into JSON because of indexing errors at the chunk boundary.
None of the following kinds of JSON trigger the error:
[ [], [], [], ... ]
[ {"a", "b"}, {"a", "b"}, {"a", "b"}, ... ]
The compressWordAsBit function does not work if there is more than one chunk. It erroneously inserts additional zeros at the end of chunks that have a size that is not a multiple of 8.