Support raw LZO (compatible with other LZO1X implementations)

[Sergei Bulgakov]

Wow, implementing #95 was all of a sudden! I've already checked it but looks like it doesn't work for me...

I have a file raw.bin with the size of 44868 bytes. I can compress it via my LZO based on the lzokay lib to a file lzo.lzo of 8672 bytes (it adds a header of 6 bytes and then delegates the compression to that lib). Uncompressing this file is possible via the original LZO implementation.

However, uncompressing it via korio lzo gives an exception. Can raw LZO be supported?

suspend fun main() {
    val raw = resourcesVfs["raw.bin"].readAll()
    println("raw size: ${raw.size}")

    val lzo = resourcesVfs["lzo.lzo"].readAll()
    println("lzo size: ${lzo.size}")

    val s = lzo.openFastStream()
    val lz = s.readBytes(2)
    val rawSize = s.readU32LE().toInt()
    val lzoData = s.readAll()

    println("${lz.map { it.toInt().toChar() }} $rawSize ${lzoData.size}")

    val rawCompressed = raw.compress(LZO)
    val size = LZO.Header().also { it.read(rawCompressed.openAsync()) }
    println("rawCompressed size: ${rawCompressed.size} $size")

    val rawCompressedUncompressed = rawCompressed.uncompress(LZO)
    println("rawCompressedUncompressed size: ${rawCompressedUncompressed.size}")

    val lzoUncompressed = lzoData.uncompress(LZO)
    println("lzoUncompressed size: ${lzoUncompressed.size}")

    val lzoUncompressedCompressed = lzoUncompressed.compress(LZO)
    println("lzoUncompressedCompressed size: ${lzoUncompressedCompressed.size}")
}

raw size: 44868
lzo size: 8672
[L, Z] 44868 8666
rawCompressed size: 89786 Header(version=4160, libVersion=8352, versionNeeded=2368, method=3, level=9, flags=0, filter=0, mode=33188, mtime=1635636518, GMTdiff=0, name=, checksum=0, uncompressedSize=44868, compressedSize=10633, checksumUncompressed=0, checksumCompressed=0)
rawCompressedUncompressed size: 44868
java.lang.IllegalStateException: INVALID LZO!
	at com.soywiz.korio.compression.lzo.LZO$Header.read(LZO.kt:36)
	at com.soywiz.korio.compression.lzo.LZO.uncompress$suspendImpl(LZO.kt:85)
	at com.soywiz.korio.compression.lzo.LZO.uncompress(LZO.kt)
	at com.soywiz.korio.compression.CompressionMethodKt.uncompress(CompressionMethod.kt:34)
	at com.soywiz.korio.compression.CompressionMethodKt$uncompress$2.invokeSuspend(CompressionMethod.kt:42)
	at com.soywiz.korio.compression.CompressionMethodKt$uncompress$2.invoke(CompressionMethod.kt)
	at com.soywiz.korio.compression.CompressionMethodKt$uncompress$2.invoke(CompressionMethod.kt)
	at com.soywiz.korio.async.RunBlockingNoSuspensionsKt.startCoroutineUndispatched(runBlockingNoSuspensions.kt:52)
	at com.soywiz.korio.async.RunBlockingNoSuspensionsKt.runBlockingNoSuspensions(runBlockingNoSuspensions.kt:19)
	at com.soywiz.korio.compression.CompressionMethodKt.uncompress(CompressionMethod.kt:41)
	at com.soywiz.korio.compression.CompressionMethodKt.uncompress(CompressionMethod.kt:49)
	at com.soywiz.korio.compression.CompressionMethodKt.uncompress$default(CompressionMethod.kt:49)
	at MainKt.main(main.kt:69)
	at MainKt$main$3.invoke(main.kt)
	at MainKt$main$3.invoke(main.kt)
	at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt$createCoroutineUnintercepted$$inlined$createCoroutineFromSuspendFunction$IntrinsicsKt__IntrinsicsJvmKt$1.invokeSuspend(IntrinsicsJvm.kt:205)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlin.coroutines.ContinuationKt.startCoroutine(Continuation.kt:115)
	at kotlin.coroutines.jvm.internal.RunSuspendKt.runSuspend(RunSuspend.kt:19)
	at MainKt.main(main.kt)

Files:
lzo.lzo.zip
raw.bin.zip

[Sergei Bulgakov]

Thank you, I've just checked and it works for me. Also, I've taken a look a bit deeper in korio and see there is LzoRawDecompressor and LzoRawCompressor. This is exactly what I need, I guess I'll use these objects directly.