npm install sha3 fails - node10/npm6

Question

npm install sha3 fails - node10/npm6

opened this issue 6 years ago · comments

Traceback:

sha3@1.2.0 install /Users/noahzinsmeister/Documents/github/smart-contracts/snowflake_poc/node_modules/sha3
node-gyp rebuild

CXX(target) Release/obj.target/sha3/src/addon.o
../src/addon.cpp:59:36: error: no matching member function for call to 'NewInstance'
info.GetReturnValue().Set(cons->NewInstance(argc, argv));
~~~~~~^~~~~~~~~~~
/Users/noahzinsmeister/.node-gyp/10.0.0/include/node/v8.h:3851:44: note: candidate function not viable: requires single argument 'context', but 2 arguments were provided
V8_WARN_UNUSED_RESULT MaybeLocal NewInstance(
^
/Users/noahzinsmeister/.node-gyp/10.0.0/include/node/v8.h:3848:44: note: candidate function not viable: requires 3 arguments, but 2 were provided
V8_WARN_UNUSED_RESULT MaybeLocal NewInstance(
^
1 error generated.
make: *** [Release/obj.target/sha3/src/addon.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23)
gyp ERR! stack at ChildProcess.emit (events.js:182:13)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:225:12)
gyp ERR! System Darwin 17.5.0
gyp ERR! command "/usr/local/Cellar/node/10.0.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/noahzinsmeister/Documents/github/smart-contracts/snowflake_poc/node_modules/sha3
gyp ERR! node -v v10.0.0
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok

Alex Tsankov · Answer 1 · Fri Apr 27 2018 03:27:19 GMT+0800 (China Standard Time)

Having the same issue.

Devin Canterberry · Answer 2 · Fri Apr 27 2018 14:40:47 GMT+0800 (China Standard Time)

Working on porting this implementation to pure JavaScript, removing the C compilation steps from installation and alleviating this issue.

Deleted user · Answer 3 · Sat Apr 28 2018 06:28:35 GMT+0800 (China Standard Time)

@canterberry any ETA on the port? I might have some time to help out this weekend. Also, will we even be able to merge the PR? The repo hasn't been updated in ages, I don't know if anyone who has push access is still around.

Devin Canterberry · Answer 4 · Sat Apr 28 2018 14:57:52 GMT+0800 (China Standard Time)

cc @FooBarWidget

Even though this particular repo has been stable for 3 years, this is a serious enough compatibility issue that it'll warrant an update to be fixed.

A direct solution for this issue might be to modernize the node-gyp layer to be compatible with Node.js 10.x which has evolved NewInstance since this library was last published. @NoahHydro This might be a good first move, just to help unblock folks who need Node.js 10.x support.

RE the JavaScript port: I'm hoping to have something working this weekend. Even though it's a port, the work is a tad ambitious, so I'm not holding my breath for it being incorporated any time in the immediate future. While I'm aiming to keep with an identical API and functionality to this library, I think the best course of action may be to develop it as a drop-in replacement for this library, with the hope of proposing to incorporate it here when it's ready.

Blade Liu · Answer 5 · Sun Apr 29 2018 16:47:58 GMT+0800 (China Standard Time)

have same issue here. wow, real big issue... How to fix?

Devin Canterberry · Answer 6 · Sun Apr 29 2018 23:24:02 GMT+0800 (China Standard Time)

@bladexxv Since this issue doesn't manifest until Node.js 10.x, a workaround would be to downgrade to Node.js 9.11.1 (the previous non-LTS release) or 8.11.1 (the current LTS release).

The minimum path to fixing this issue altogether in Node.js 10.x would be:

A maintainer of this repo would need to merge #33.
A maintainer of this repo would need to increment the version number and tag a new release.
A maintainer with access to publish the sha3 package to npm would need to publish the new patch version to npm.
Direct dependencies on this project would need to be updated to (a) specify the new version, and/or (b) update the lockfile to specify the new patch version.

fratellopartigiano · Answer 7 · Mon Aug 20 2018 21:58:06 GMT+0800 (China Standard Time)

It fails with

gyp ERR! node -v v9.2.0
gyp ERR! node-gyp -v v3.8.0

too.

Max Peng · Answer 8 · Tue Aug 21 2018 08:14:12 GMT+0800 (China Standard Time)

Have the same issue with:

gyp ERR! node -v v10.9.0
gyp ERR! node-gyp -v v3.7.0

After I downgrade the node version to 8.11.1 as @canterberry suggested, it works.

Paul Melero · Answer 9 · Thu Mar 28 2019 22:00:07 GMT+0800 (China Standard Time)

gyp info using node-gyp@3.8.0
gyp info using node@10.15.3
gyp info spawn C:\Python27\python.EXE

Luis Schliesske · Answer 10 · Mon May 06 2019 23:45:51 GMT+0800 (China Standard Time)

Hi everyone,
I posted this under this issue in the ethereum/web3.js repo. But not sure whether it is actually more relevant here, so here goes again:
Suddenly since yesterday I cannot npm install web3 any more. I am using MacOs and hombrew node. The only change that might have caused this is that I think I ran a brew update and brew upgrade recently.
npm -v 6.9.0
node -v v12.1.0

The problem seems to have something to do with sha3 and node-gyp. I saw this issue which seems to be related. Can anyone please help me out or point me to a solution for macOS with my node and npm versions?

error message:
10 warnings and 12 errors generated.
make: *** [Release/obj.target/sha3/src/addon.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack at ChildProcess.emit (events.js:196:13)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:256:12)
gyp ERR! System Darwin 18.5.0
gyp ERR! command "/usr/local/Cellar/node/12.1.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/xyz/Desktop/udemy/udemy-eth-sol/projects/kickstart/node_modules/sha3
gyp ERR! node -v v12.1.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm WARN ajv-keywords@2.1.1 requires a peer of ajv@^5.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN kickstart@1.0.0 No description
npm WARN kickstart@1.0.0 No repository field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: scrypt@6.0.3 (node_modules/scrypt):
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: scrypt@6.0.3 install: node-gyp rebuild
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: Exit status 1

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! sha3@1.2.2 install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the sha3@1.2.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Devin Canterberry · Answer 11 · Tue May 07 2019 01:06:20 GMT+0800 (China Standard Time)

Yeah, the problem is that node-gyp is not compatible with Node.js 12.x, which means the 1.x branch of this project (and any other project that uses native C compilation steps) is incompatible with Node.js 12.x. Unfortunately, older versions of web3 still depend on this old implementation.

As a workaround, you can try npm install sha3@2.0.2 in your project, which may coerce web3 to resolve its sha3 dependency to this new version. Version 1.x and 2.x of this package are interface-compatible, so there should be no compatibility issues with web3 in doing so.

Luis Schliesske · Answer 12 · Tue May 07 2019 23:58:23 GMT+0800 (China Standard Time)

Hi @canterberry thank you for offering your help. Unfortunately npm install sha3@2.0.2 into my project directory did not resolve the issue with npm install web3 for me. I am relatively new to node, so no idea if this makes sense, but could I somehow use a separate older version of node in this project directory, to . be able to npm install web3 again?
The npm install errors with:
10 warnings and 12 errors generated.
make: *** [Release/obj.target/sha3/src/addon.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: make failed with exit code: 2
gyp ERR! stack at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack at ChildProcess.emit (events.js:196:13)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:256:12)
gyp ERR! System Darwin 18.5.0
gyp ERR! command "/usr/local/Cellar/node/12.1.0/bin/node" "/usr/local/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/xyz/Desktop/udemy/udemy-eth-sol/projects/kickstart/node_modules/keccakjs/node_modules/sha3
gyp ERR! node -v v12.1.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok
npm WARN ajv-keywords@2.1.1 requires a peer of ajv@^5.0.0 but none is installed. You must install peer dependencies yourself.
npm WARN kickstart@1.0.0 No description
npm WARN kickstart@1.0.0 No repository field.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! sha3@1.2.2 install: node-gyp rebuild
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the sha3@1.2.2 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

Devin Canterberry · Answer 13 · Wed May 08 2019 00:41:18 GMT+0800 (China Standard Time)

Ah, I was hoping it was that easy. Apparently not!

Have a look at the npm-force-resolutions library, or consider using Yarn which natively supports selective version resolutions. Looks like with either of these methods, it's almost as easy to force a resolution as my initial suggestion.

(TIL how to force transitive dependency resolutions in npm and yarn.)

TL;DR:

Add the following block to your package.json:

"resolutions": {
  "sha3": "2.0.2"
}

Run rm -fR node_modules to clear out your project's resolved dependencies.
Run npx npm-force-resolutions to patch your package-lock.json.
Run npm install again.

Luis Schliesske · Answer 14 · Wed May 08 2019 16:03:03 GMT+0800 (China Standard Time)

Hi @canterberry. Thanks again for your efforts. I followed your guidance and

added this block to my package.json
"resolutions": { "sha3": "2.0.2" }
I ran rm -fR node_modules
Ran npx npm-force-resolutions
Ran npm install web3 again

still get the same error message unfortunately. Anything else you can spot here before I move over to Yarn? Thanks for all the help again.

Devin Canterberry · Answer 15 · Thu May 09 2019 02:51:37 GMT+0800 (China Standard Time)

In trying to reproduce this locally (i.e: installing web3 from scratch in an empty project using Node.js 12.1.0 via a simple npm install web3), I get an node-gyp rebuild error on the scrypt@6.0.3 package. That package hasn't been updated in 3 years, so even if you were able to get the sha3 package resolved to 2.x, that version of web3 would still have a hard dependency on an scrypt package with no version available that works on Node.js 12.x.

The steps you followed look correct, but I tried following them, myself, and saw this warning in the console:

warning Resolution field "sha3@2.0.2" is incompatible with requested version "sha3@^1.1.0"

Apparently both npm and yarn enforce fuzzy version selectors even when attempting to force a resolution via resolutions.

There doesn't appear to be a workaround, then, for using web3 on Node.js 12.x. The only solution may be to submit a PR to web3 that migrates its dependencies to alternatives (or newer versions) which do not depend on the node-gyp toolchain.

Devin Canterberry · Answer 16 · Thu May 09 2019 06:17:31 GMT+0800 (China Standard Time)

@bytezantium v1.2.3 has been released. Please give that a shot (via the same steps outline above). Since 1.2.3 should match the fuzzy version selector ^1.1.0, the forced resolution should work.

Luis Schliesske · Answer 17 · Sun May 26 2019 22:47:40 GMT+0800 (China Standard Time)

Hi @canterberry . Sorry I only went back now to check again. You are the real MVP for taking all this time helping and fixing this. So bad news: it still wont work with web3@1.0.0-beta.35 - do you know why? GOOD NEWS: it does indeed work now with latest web3 (beta.55). Thanks so much!

Devin Canterberry · Answer 18 · Tue May 28 2019 02:59:01 GMT+0800 (China Standard Time)

😄 Thanks! Dealing with the Rube Goldberg of the npm dependency graph can be rough, so thanks for your patience. I'm confident that I've done what I can, within the scope of this package, for compatibility with all (known) environments. The rest depends on maintainers of the individual packages. web3@1.0.0-beta.35 is unlikely to ever work in Node.js 12.x because of its dependency on scrypt, which appears to have been abandoned for several years. Since then, web3 seems to have removed its dependency on this package (sha3), and has made scrypt an optional dependency (?) so that its inevitable build failure on Node.js 12.x no longer causes the web3 install itself to fail.

Luis Schliesske · Answer 19 · Thu May 30 2019 21:08:36 GMT+0800 (China Standard Time)

thanks a lot for your input on this!

Juan C · Answer 20 · Wed Jul 24 2019 08:17:54 GMT+0800 (China Standard Time)

I followed @canterberry steps, but it didn't work in my case unfortunately. I was forced to downgrade node to 11.13.0

Steps I followed to downgrade from node 12 to 11:

sudo npm install -g n install a program called n
sudo n 11.13.0 downgrading node to 11.13.0
rm -fR node_modules cleaning dependencies
npm install reinstall dependencies

BTW - I am on mac

Devin Canterberry · Answer 21 · Wed Jul 24 2019 08:47:25 GMT+0800 (China Standard Time)

The latest version, sha3@1.2.3 (aka: sha3@native) should compile on Node.js 12.x. Versions prior to 1.2.3 will not work, and will require a forced resolution (or delete and regenerate the lockfile for your version manager of choice).

@Juan-cc Can you provide some more details for your use case? Namely:

Are you using this sha3 library directly, or is it a transitive dependency of something else you are using? If the latter, which project is it?
What are the symptoms you are running into? Is it the node-gyp build failure, or maybe some other issue?
In which version of Node.js are you seeing the issue?
Can you share the build output?
Can you link to or share your lockfile (package-lock.json and/or yarn.lock)?

Juan C · Answer 22 · Thu Jul 25 2019 05:52:37 GMT+0800 (China Standard Time)

@canterberry please find my answers below

I am not using sha3 directly. It's through project https://github.com/PaulRBerg/confidential-tokens?source=post_page---------------------------
Yes it's node-gyp rebuild failure.
Node 12.7.0

5. package-lock.json -> https://file.io/8Bftky

Devin Canterberry · Answer 23 · Thu Jul 25 2019 06:27:56 GMT+0800 (China Standard Time)

Strange -- in the screenshot for (4), I see the path is "/usr/local/Cellar/node/12.6.0/bin/node", but below I can see node -v v12.7.0. Probably unrelated, but an odd mismatch to see.

In attempting to reproduce this locally, but on Linux, I see the following:

$ npm install sha3@1.2.3

> sha3@1.2.3 install /path/to/example/node_modules/sha3
> node-gyp rebuild

make: Entering directory '/path/to/example/node_modules/sha3/build'
  CXX(target) Release/obj.target/sha3/src/addon.o
../src/addon.cpp: In static member function ‘static Nan::NAN_METHOD_RETURN_TYPE SHA3Hash::New(Nan::NAN_METHOD_ARGS_TYPE)’:
../src/addon.cpp:49:23: warning: ‘new’ of type ‘SHA3Hash’ with extended alignment 32 [-Waligned-new=]
    obj = new SHA3Hash();
                       ^
../src/addon.cpp:49:23: note: uses ‘void* operator new(std::size_t)’, which does not have an alignment parameter
../src/addon.cpp:49:23: note: use ‘-faligned-new’ to enable C++17 over-aligned new support
../src/addon.cpp: In static member function ‘static void SHA3Hash::Init(Nan::ADDON_REGISTER_FUNCTION_ARGS_TYPE)’:
../src/addon.cpp:83:27: warning: ‘bool v8::Object::Set(v8::Local<v8::Value>, v8::Local<v8::Value>)’ is deprecated: Use maybe version [-Wdeprecated-declarations]
   target->Set(className, f);
                           ^
In file included from /path/to/home/.node-gyp/12.7.0/include/node/v8-internal.h:14:0,
                 from /path/to/home/.node-gyp/12.7.0/include/node/v8.h:25,
                 from /path/to/home/.node-gyp/12.7.0/include/node/node.h:63,
                 from ../src/addon.cpp:1:
/path/to/home/.node-gyp/12.7.0/include/node/v8.h:3366:26: note: declared here
                     bool Set(Local<Value> key, Local<Value> value));
                          ^
/path/to/home/.node-gyp/12.7.0/include/node/v8config.h:326:3: note: in definition of macro ‘V8_DEPRECATE_SOON’
   declarator __attribute__((deprecated(message)))
   ^~~~~~~~~~
  CXX(target) Release/obj.target/sha3/src/displayIntermediateValues.o
  CXX(target) Release/obj.target/sha3/src/KeccakF-1600-reference.o
  CXX(target) Release/obj.target/sha3/src/KeccakNISTInterface.o
  CXX(target) Release/obj.target/sha3/src/KeccakSponge.o
  SOLINK_MODULE(target) Release/obj.target/sha3.node
  COPY Release/sha3.node
make: Leaving directory '/path/to/example/node_modules/sha3/build'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN example@1.0.0 No description
npm WARN example@1.0.0 No repository field.

+ sha3@1.2.3
added 2 packages from 9 contributors and audited 2 packages in 2.174s
found 0 vulnerabilities

$ node --version
v12.7.0
$ npm --version
6.10.0

This does not produce a build failure for me, but I am seeing a deprecation warning.

In Node.js 12.3.0, I do see a build error:

gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/path/to/node/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:200:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:272:12)
gyp ERR! System Linux 4.15.0-55-generic
gyp ERR! command "/path/to/bin/node" "/path/to/node/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /path/to/example/node_modules/sha3
gyp ERR! node -v v12.3.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

Node.js 12.4.0, 12.5.0, and 12.6.0, however, appear to build without errors (although the deprecation warnings are still present).

I have (and continue) to strongly advise package maintainers to migrate to v2.x of this package, which is a pure JavaScript implementation that does not suffer from drift in the node-gyp library and Python/C build toolchain and thus does not suffer from intermittent build failures due to changes in the Node.js and node-gyp toolchain.

Devin Canterberry · Answer 24 · Thu Jul 25 2019 06:50:00 GMT+0800 (China Standard Time)

Created #60 to capture this defect, as it represents a different occurrence of this symptom.

Devin Canterberry · Answer 25 · Thu Jul 25 2019 07:49:32 GMT+0800 (China Standard Time)

@Juan-cc In re-reading this topic, I noticed you downgraded to Node.js v11.x. Fair warning, this version of Node.js is no longer supported by the Node.js team. v12.x, v10.x, and v8.x all remain currently active, though.

I've opened PR #61 to start down the path toward resolving the deprecation warnings, and hopefully toward resolving your issue and that of others who may be experiencing it.

Devin Canterberry · Answer 26 · Thu Jul 25 2019 08:10:33 GMT+0800 (China Standard Time)

@Juan-cc It looks like there are no deprecation warnings or errors in the Travis CI build on Node.js 12.7.0. The build output looks sparkly clean across all supported Node.js versions, but I'm definitely seeing different results locally. This indicates a compatibility issue with the C++ compiler rather than with Node.js, the V8/add-ons API, or node-gyp.

Which C compiler are you using? (i.e: the output of cc --version and/or gcc --version)

Travis CI: (no errors or warnings)

gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

My local environment: (deprecation warnings, no errors)

gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0

Juan C · Answer 27 · Thu Jul 25 2019 19:05:50 GMT+0800 (China Standard Time)

Hi @canterberry , first of all thanks for taking the time to help with this issue.
Please find below the info requested:

juan@Juan-Mac  ~  cc --version                          ✔  1847  12:57:58
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

 juan@Juan-Mac  ~  gcc --version                                                    ✔  1848  12:58:15
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

philipkd · Answer 28 · Tue Jul 23 2024 06:36:17 GMT+0800 (China Standard Time)

I tried a bunch of node resolutions, tracing through package-lock.json and comparing with popular versions of packages on npm. This worked for me on node 18:

  "resolutions": {
    "eth-lib": "0.2.8"
  }

as well as adding this to scripts: "preinstall": "npx npm-force-resolutions"
then rm -R node_modules/* and npm i as usual