clang-12 + Release mode causes zlib failures (apparent data corruption)
djmarcin opened this issue · comments
Version
16.3.0, 16.4.0, 16.4.2
Platform
Linux 5.8.0-53-generic #60~20.04.1-Ubuntu SMP Thu May 6 09:52:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Subsystem
zlib
What steps will reproduce the bug?
Install clang 12.
$ sudo apt install clang-12
$ clang-12 --version
Ubuntu clang version 12.0.1-++20210605063018+0826268d59c6-1~exp1~20210605043737.99
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Download and build Node with clang-12
$ curl -sL https://github.com/nodejs/node/archive/refs/tags/v16.4.2.tar.gz | tar xz
$ cd node-16.4.2 && export CC="clang-12" CXX="clang++-12" && ./configure && make -j$(nproc) BUILDTYPE=Release V=0 -C out
Run the following script with the produced binary:
$ curl -o /tmp/prettier.tgz https://registry.npmjs.org/prettier/-/prettier-2.2.1.tgz
$ cat - << EOF | out/Release/node
const fs = require('fs');
const { createGunzip } = require('zlib');
const { pipeline } = require('stream');
let inFile = fs.createReadStream('/tmp/prettier.tgz');
let outFile = fs.createWriteStream('/tmp/prettier.tar');
pipeline(inFile, createGunzip(), outFile, (err) => {
if (err) {
console.error('An error occurred:', err);
process.exitCode = 1;
}
});
EOF
Observe output:
An error occurred: Error: incorrect data check
at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
errno: -3,
code: 'Z_DATA_ERROR'
}
How often does it reproduce? Is there a required condition?
Always when compilied in Release mode with clang-12
What is the expected behavior?
prettier.tar should be produced without errors
What do you see instead?
An error occurred: Error: incorrect data check
at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
errno: -3,
code: 'Z_DATA_ERROR'
}
Additional information
This error does not appear when:
- Compiling with BUILDTYPE=Debug and using out/Debug/node
- Compiling with clang-11 (tested with
Ubuntu clang version 11.0.0-2~ubuntu20.04.1) - Compiling with GCC (tested with
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0)
This error is also not present when compiling upstream zlib (https://zlib.net/) with clang-12.
This issue breaks npm install for prettier and several other packages as well because the files cannot be decompressed.
Built on an AMD Threadripper CPU.
Refs: vercel/pkg-fetch#208
I can reproduce this issue with Apple Clang compiled macos-x64 executable.
clang -v
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin21.0.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
./configure && make
At least Node 14 and 16 release lines are impacted.
Interestingly, the issue does not appear with arm64 executable compiled with the same toolchain.
This could become a release-critical bug, once the new Clang/Xcode versions roll in. I suggest that we re-triage this issue as zlib, confirmed-bug.
I am looking at the issue right now, and to raise attention here are some pings to recent committers to the zlib parts: @RaisinTen, @addaleax, @mscdex, @aduh95, @ronag .
Perhaps we need to apply the fix referred to in #31800 (comment) to solve this issue. But we need #33044 to land first to be able to update our copy of zlib.
Newer version of clang (tested on macOS with Xcode 13.2.1) seem to fail to generate correct code when compiling zlib/contrib/optimizations/inffast_chunk.c with -O2 or higher optimization. -O1 seems to work fine.
#45387 landed. I cannot reproduce on macOS with clang 14. Can this be closed?
Did the fix find its way into all currently supported versions of node.js?
Did the fix find its way into all currently supported versions of node.js?
#45387 is not yet included in any version.
I suggest to close the bug after the upstream fix has been merged into all currently supported versions of node.js.
Why? we usually close the issue when the fix lands on main. If the issue is fixed on main there is no reason to keep it open.
I can confirm that the bug no longer exists on main. So, you might close this if you want. I still need to keep my workaround in my build script for releases until all releases are fixed.
Thank you for the confirmation.
I experience this issue with clang-14, nvm, node 16.15.1, npm 8.11.0, macOS 13.3.1 (22E261). Using nvm install v16.15.1 --shared-zlib fixed the issue. npm install would fail with Error code Z_DATA_ERROR Zlib: incorrect data check -3. I followed this instructions to fix it