nodejs / node

Node.js JavaScript runtime :sparkles::turtle::rocket::sparkles:

Home Page:https://nodejs.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

clang-12 + Release mode causes zlib failures (apparent data corruption)

djmarcin opened this issue · comments

Version

16.3.0, 16.4.0, 16.4.2

Platform

Linux 5.8.0-53-generic #60~20.04.1-Ubuntu SMP Thu May 6 09:52:46 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

zlib

What steps will reproduce the bug?

Install clang 12.

$ sudo apt install clang-12
$ clang-12 --version
Ubuntu clang version 12.0.1-++20210605063018+0826268d59c6-1~exp1~20210605043737.99
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Download and build Node with clang-12

$ curl -sL https://github.com/nodejs/node/archive/refs/tags/v16.4.2.tar.gz | tar xz
$ cd node-16.4.2 && export CC="clang-12" CXX="clang++-12" && ./configure && make -j$(nproc) BUILDTYPE=Release V=0 -C out

Run the following script with the produced binary:

$ curl -o /tmp/prettier.tgz https://registry.npmjs.org/prettier/-/prettier-2.2.1.tgz
$ cat - << EOF | out/Release/node
const fs = require('fs');
const { createGunzip } = require('zlib');
const { pipeline } = require('stream');
let inFile = fs.createReadStream('/tmp/prettier.tgz');
let outFile = fs.createWriteStream('/tmp/prettier.tar');
pipeline(inFile, createGunzip(), outFile, (err) => {
  if (err) {
    console.error('An error occurred:', err);
    process.exitCode = 1;
  }
});
EOF

Observe output:

An error occurred: Error: incorrect data check
    at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
  errno: -3,
  code: 'Z_DATA_ERROR'
}

How often does it reproduce? Is there a required condition?

Always when compilied in Release mode with clang-12

What is the expected behavior?

prettier.tar should be produced without errors

What do you see instead?

An error occurred: Error: incorrect data check
    at Zlib.zlibOnError [as onerror] (node:zlib:191:17) {
  errno: -3,
  code: 'Z_DATA_ERROR'
}

Additional information

This error does not appear when:

  • Compiling with BUILDTYPE=Debug and using out/Debug/node
  • Compiling with clang-11 (tested with Ubuntu clang version 11.0.0-2~ubuntu20.04.1)
  • Compiling with GCC (tested with gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0)

This error is also not present when compiling upstream zlib (https://zlib.net/) with clang-12.

This issue breaks npm install for prettier and several other packages as well because the files cannot be decompressed.

Built on an AMD Threadripper CPU.

Refs: vercel/pkg-fetch#208

I can reproduce this issue with Apple Clang compiled macos-x64 executable.

clang -v
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin21.0.0
Thread model: posix
InstalledDir: /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
./configure && make

At least Node 14 and 16 release lines are impacted.

Interestingly, the issue does not appear with arm64 executable compiled with the same toolchain.

This could become a release-critical bug, once the new Clang/Xcode versions roll in. I suggest that we re-triage this issue as zlib, confirmed-bug.

I am looking at the issue right now, and to raise attention here are some pings to recent committers to the zlib parts: @RaisinTen, @addaleax, @mscdex, @aduh95, @ronag .

Perhaps we need to apply the fix referred to in #31800 (comment) to solve this issue. But we need #33044 to land first to be able to update our copy of zlib.

Newer version of clang (tested on macOS with Xcode 13.2.1) seem to fail to generate correct code when compiling zlib/contrib/optimizations/inffast_chunk.c with -O2 or higher optimization. -O1 seems to work fine.

#45387 landed. I cannot reproduce on macOS with clang 14. Can this be closed?

Did the fix find its way into all currently supported versions of node.js?

Did the fix find its way into all currently supported versions of node.js?

#45387 is not yet included in any version.

I suggest to close the bug after the upstream fix has been merged into all currently supported versions of node.js.

Why? we usually close the issue when the fix lands on main. If the issue is fixed on main there is no reason to keep it open.

I can confirm that the bug no longer exists on main. So, you might close this if you want. I still need to keep my workaround in my build script for releases until all releases are fixed.

Thank you for the confirmation.