N-API support

Question

N-API support

NickNaso opened this issue 6 years ago · comments

Node 8 has a new experimental feature called N-API which is aimed at reducing maintenance cost for node native addons. Check out this blog for more details on its benefits.

farmhash is a popular module, and in order to help the Node.js community make the important transition to N-API, we are hoping you will be able to work with us in order to migrate farmhash to N-API. Your support and feedback is important in making this effort a success.

We (N-API team) have built a preliminary port of farmhash on top of N-API as part of our effort to understand the API surface used by native modules. We used this port and the port of some other modules to determine which areas to focus on converting to N-API.
This port has all of the NAN code removed and replaced by N-API equivalents. While it passes all the unit tests, it might need a bit more refinement before getting merged in.
Please take a look if you're interested. I will be happy to answer questions about the port, point to documentation/code, etc. I'm also happy to open a PR if you're happy with the port in it's current shape.

If you like, you can join our WG meeting which happens every Thursday at 1:30 Eastern / 10:30 Pacific US time, to discuss any issues or ask us any questions.

If you don't want to take a dependency on a feature that still has "experimental" status, we understand, and a good option might be to maintain a branch that uses N-API, that is kept mostly up to date with development occurring in farmhash/master. We have some precedence to follow with node.bcrypt.js which recently published a N-API version of their module. Of course we'd appreciate feedback about anything we can do to improve the development or migration experience.

You can find my work here: https://github.com/NickNaso/farmhash/tree/napi

I will be happy to contribute and help in any way.

Lovell Fuller · Answer 1 · Fri Mar 16 2018 21:48:14 GMT+0800 (China Standard Time)

Buon giorno, it looks like N-API has just moved from "experimental" to "stable" ahead of Node 10, which is great news. Thank you for taking the time to demonstrate how easy migrating to it can be.

The primary benefit of N-API to end users, and therefore module maintainers, is the avoidance of NODE_MODULE_VERSION mismatch errors when upgrading major versions. (I'm amazed there are fewer than 3k issues about this across the whole of GitHub.)

Is my understanding of the N-API experience for end-users of each major Node LTS version correct?

Node 4 and 6 users get a polyfill and are therefore still tied to their respective NODE_MODULE_VERSION.
Node 8 users get "Warning: N-API is an experimental feature and could change at any time."
Node 10 users will start to see the benefits when they upgrade to the future Node 12.

Other questions:

Will the "stable" N-API arriving in Node 10 be backported to Node 8 to remove the warning?
Is there information about expected performance differences between N-API and nan, if any?
I see node-pre-gyp has support for N-API. Are there plans to make a similar change to the prebuild family upon which farmhash depends?
I have other native modules that will need the nan to N-API treatment. In your experience what sort of things is the conversion tooling unable to deal with?

Nicola Del Gobbo · Answer 2 · Sat Mar 17 2018 03:39:16 GMT+0800 (China Standard Time)

Hi @lovell ,
yes N-API will be out of experimental in the next release of Node.js. It guarantees the API and ABI compatibility so this means that you do not need to recompile your native addon if you switch between different versions of Node.js even if you are using different JavaScript engine (V8 or ChakraCore). It's great for both end user and mantainers. All the LTS 4 and 6 have the polyfill and if you use node-addon-api the N-API is automatically included if your Node.js version hasn't it.
Now we worked to exit from the experimental status and in next months we planned to work on backport of N-API to 6.x and 8.x.
In our experiment all modules worked well from version 4 to 8, but we understand that we need to investigate this more deeply.
In terms of performance I have good news it seems tat N-API modules are a little faster than the equivalent module written using nan, but in most cases they are equivalent. This is a good result considering the abstraction layer represented by N-API.
In our porting we experimented some performance issue but they have been solved at the end.
We worked first on node-pre-gyp support, but you can follow this issue nodejs/abi-stable-node#291 to know about prebuild support.
The conversion tools helps to create the scaffolding of your project:

remove nan and substitute with node-addon-api on your package.json and gyp file
substitute all the nan code with the right class and object of the N-API

At the end you will need to write some part of your code.
We also are working on documentation and I hope that in the next two weeks it will be completed.

I hope that my answer is exhaustive.

About farmhash if you want experiment with N-API you can create a branch and call it "napi" so I can execute the PR on it. After that we can iterate to over it and at the end publish a tagged version of farmhash like reported here: https://nodejs.org/en/docs/guides/publishing-napi-modules/

Lovell Fuller · Answer 3 · Sat Mar 17 2018 04:53:54 GMT+0800 (China Standard Time)

That's a very comprehensive reply Nicola, molte grazie.

I think I'd like to see the following before a release of farmhash based on N-API.

The stable N-API is backported to Node 8 so it no longer produces the warning.
The prebuild tooling supports N-API.

I've created your suggestion of a napi work-in-progress branch for this; PRs welcome.

Lovell Fuller · Answer 4 · Sun Mar 18 2018 00:51:01 GMT+0800 (China Standard Time)

Thanks for the PR.

Anyone that would like to test this can do so with:

npm install lovell/farmhash#napi

I've subscribed to the highlighted Node 8 and prebuild issues so as soon as they're marked as closed we can merge this and release. I also plan to do some performance testing.

Nicola Del Gobbo · Answer 5 · Mon Oct 08 2018 11:20:32 GMT+0800 (China Standard Time)

Hi @lovell I want only notify that last week prebuild added the support for N-API https://www.npmjs.com/package/prebuild#n-api-considerations.

Lovell Fuller · Answer 6 · Mon Oct 08 2018 16:38:48 GMT+0800 (China Standard Time)

@NickNaso Thanks, that's great news and I saw that N-API is considered stable in Node 8 too.

Happy for a PR with the extra prebuild config if you're able or will get around it myself at some point.

Lovell Fuller · Answer 7 · Thu Jun 20 2019 05:24:40 GMT+0800 (China Standard Time)

I've merged the napi branch to master - will update config/deps etc and publish to npm in the next few days - thanks again for your work on this Nick.

Nicola Del Gobbo · Answer 8 · Thu Jun 20 2019 05:59:16 GMT+0800 (China Standard Time)

Great news I'm very happy to hear this.

Lovell Fuller · Answer 9 · Sat Jun 22 2019 05:59:52 GMT+0800 (China Standard Time)

@NickNaso Are you able to help with a build failure with N-API and Node v8 (v10+ is OK), e.g. https://travis-ci.org/lovell/farmhash/jobs/548869980 ?

Lovell Fuller · Answer 10 · Sun Jun 23 2019 19:01:28 GMT+0800 (China Standard Time)

It looks like the build failure is due to prebuild-ci using node-abi to determine that the value of --target for Node ABI 57 is 8.0.0, an approach that works for nan-based modules.

I think this means that napi-based modules have to use a more recent version in the --target flag. Is there a minimum minor/patch version of Node 8 required for napi?

Nicola Del Gobbo · Answer 11 · Sun Jun 23 2019 19:40:01 GMT+0800 (China Standard Time)

Hi @lovell,
sorry for the late in response. If I remember correctly we introduced napi_async_context in Node.js version 8.6.0.
When you build N-API module all that you need is to specify the N-API version:
prebuild --target 3 --runtime napi because you should be independent from the NODE_MODULE_VERSION.

Lovell Fuller · Answer 12 · Sun Jun 23 2019 21:52:50 GMT+0800 (China Standard Time)

That makes sense, thanks for confirming, I'll need to add support for --runtime napi to the prebuild-ci module.

Nicola Del Gobbo · Answer 13 · Tue Jun 25 2019 00:46:34 GMT+0800 (China Standard Time)

I want to add this information:
Until the Node.js version 8.11.1 N-API is at version 1 and from the Node.js version 8.11.2 N-API is at version 3.
If you need to know the N-API version you can use this module napi-build-utils that is the same that prebuild use internally.

Lovell Fuller · Answer 14 · Mon Jul 01 2019 03:22:34 GMT+0800 (China Standard Time)

Thanks for your help Nick, it's been a bit of a learning exercise for me, e.g. I had to update prebuild-ci to add support for the napi runtime at prebuild/prebuild-ci#18

Anyway, v3.0.0 is now published with prebuilt binaries for N-API ABI version 3, supported by both Node 10 and 12 (and some more recent versions of Node 8, but have decided to drop support for that), with a fallback to the Node ABIs should N-API fail for any reason at install time. Woo!