Please release 1.8.5

Question

Please release 1.8.5

wdroste opened this issue 2 years ago · comments

There's a critical bug in #292 / #293 that's breaking one of our applications. It looks like the last attempt at release failed can you please resolve this.

Daniel Fernández · Answer 1 · Fri Dec 03 2021 00:47:18 GMT+0800 (China Standard Time)

This is critical for us too. Any attempts to use pure-java on a snappy-compressed file end up with a java.lang.IllegalArgumentException: newLimit > capacity being thrown due to the uncompressed buffer being resized beyond its capacity due to the wrong use of uncompressed.position() fixed in #293.

Please release 1.1.8.5 as soon as possible.

Will Droste · Answer 2 · Tue Dec 07 2021 04:31:59 GMT+0800 (China Standard Time)

@xerial can you please update this. It's been since Oct 20th this fix has been in.

Devaraj Kavali · Answer 3 · Wed Jun 08 2022 06:13:08 GMT+0800 (China Standard Time)

we are also facing this issue, any plans to release this bug fix.

Thomas Decaux · Answer 4 · Sat Jul 02 2022 15:54:41 GMT+0800 (China Standard Time)

Hello @xerial , thanks you for this library, any update on this one please?

I didnt find any workaround to make it work with Spark 3.2.1 on kubernetes.

Thanks you very much!

Jiannan Zhang · Answer 5 · Thu Sep 08 2022 19:34:06 GMT+0800 (China Standard Time)

@xerial Hi,thank you for this library.
This bug has been fixed for a year.
Would be grateful if 1.1.8.5 could be released.

Brais Couce · Answer 6 · Wed Oct 12 2022 00:36:22 GMT+0800 (China Standard Time)

Hi,

Do you know when the version with this fix will be released?

Regards.

Taro L. Saito · Answer 7 · Wed Oct 12 2022 00:57:30 GMT+0800 (China Standard Time)

#293 introduces a binary change, and it will not work in some platforms like AIX/ppc,ppc64, s390x, etc., which are not covered by docker-based cross-compilers.

If @odaira can rebuild native libraries for these platforms as he did 4 years ago, it's safe to release. Otherwise, we need to drop support for these platforms.

Taro L. Saito · Answer 8 · Wed Oct 12 2022 00:59:07 GMT+0800 (China Standard Time)

Pure-java support was an attempt to remove such a dependency on human labor, but it seems the current pure-java snappy code is not properly implemented and caused another issue, so it's blocking the project to ship a new release.

Daniel Fernández · Answer 9 · Tue Oct 18 2022 17:32:11 GMT+0800 (China Standard Time)

@xerial The change in #293 is meant to affect the pure-java implementation only, and it simply refactors the moment in which the size of the compressed/uncompressed buffers is computed, so that the offset used as argument at the limit operation is not wrongly affected by the previous compression/uncompression operation.

Could you please elaborate on how this fix affects the AIX/ppc, ppc64, etc. environments? Do you mean that, by applying it, only the pure-java implementation would be fixed but not the native libraries? And if so... wouldn't it be possible to release 1.1.8.5 with this fix only, and keep a ticket open for the platforms that cannot benefit from this change?

This library is used by the Apache Hadoop libraries for snappy compression/decompression of large data files in data lakes, which means a really huge amount of corporate users. But in practice this issue makes it unusable in production, forcing anyone who needs to use this in a stable environment to keep their own fork 😞

Taro L. Saito · Answer 10 · Sun Jan 29 2023 00:16:45 GMT+0800 (China Standard Time)

Released https://github.com/xerial/snappy-java/releases/tag/v1.1.9.0, which no longer supports pure-java mode. Some native libraries for special platforms like AIX are not included.