Differenciate between armv6 and armv7
piranna opened this issue · comments
As commented at nodejs/node#13629 (comment), there's no way to differenciate between armv6 and armv7 prebuild images, all of them have the arm arch. One solution we could implement here is to allow to set armv6 and armv7 as arch, being arm an alias for one of them, probably the simpler one (armv6) this is alike the situation with GCC compiler were targeted just as x86 for broadest compatibility, being first an alias for i386, later i486 and now i686.
For this to work on prebuild, new binaries would need to be uploaded with the de-aliased arch. Then, prebuild-install would try to download first the de-aliased arch too, and if it finds a 404 error, try to download the aliased one (arm) in case it's an old prebuild image.
That's exactly the problem I am facing at the moment with shipping prebuilt binaries of zeromq.
Let's dive into this? Should be easy to guess which arm version is used, as there is an option at process.config.variables.arm_version.
So we could look into this option if process.arch equals to arm.
Should be easy to guess which arm version is used, as there is an option at process.config.variables.arm_version.
Are you talking about prebuild-install? I was thinking about de-aliasing by a default value as GCC does, but yes, it makes sense to de-aliase when downloading the prebuild image to the current architecture of the Node.js binary if not set explicitly :-)
Both prebuild and prebuild-install would need to go in parallel on this point, but probably work on downloading with prebuild-install can go first so we can test first current prebuild images with the fallback mechanism, and when we have it working and tested move to prebuild and compilation and upload process, how do you see it? Can you be able to take a look on that? I can be a beta-tester with NodeOS prebuild images...
That sounds like a plan, but I would have to dive into this code first. Would be great to get a response to this idea from the project owners.
I agree. I would do it myself, but I'm too busy lately. I could be able to give you some indications through. Arch selection is done at https://github.com/mafintosh/prebuild-install/blob/master/rc.js#L28, so you'll only need to wrap it to de-aliase it. Checking for 404 error and retry again with the arm legacy arch needs to be done at https://github.com/mafintosh/prebuild-install/blob/master/download.js#L68. prebuild-install code is easy to understand, don't need to look for more :-)
Hello, a possible alternative solution for ARM version differentiation might be to add an explicit --arm-version flag. This would support not only ARMv6 vs ARMv7 but also future-proof against things like ARMv8 vs ARMv8.3 (the latter provides a JavaScript-specific FJCVTZS instruction).
With prebuild-install the proposed --arm-version flag would be a boolean flag that, if provided/true, would ensure {arch} has the version suffix on ARM devices.
| process.arch | process.config.variables.arm_version | --arm-version | {arch} |
|---|---|---|---|
| arm | 6 | false (default) | arm |
| arm | 7 | false (default) | arm |
| arm64 | 8 | false (default) | arm64 |
| arm | 6 | true | armv6 |
| arm | 7 | true | armv7 |
| arm64 | 8 | true | arm64v8 |
With prebuild the use of cross-compilation is highly likely for ARM binaries so the proposed --arm-version flag would accept the specific ARM version being built as a numeric value.
CXX=arm-rpi-linux-gnueabihf-g++ prebuild --arch=arm --arm-version=6 ...
CXX=arm-linux-gnueabihf-g++ prebuild --arch=arm --arm-version=7 ...
CXX=aarch64-linux-gnu-g++ prebuild --arch=arm64 --arm-version=8 ...A quick scan of popular modules that depend on prebuild-install (e.g. bufferutil, utf-8-validate, microtime, snappy, leveldown, iltorb) reveals only one that currently ships binaries for ARM, namely zeromq. It uses cross-compilation and contains custom, pre-prebuild(-install) logic to differentiate between ARM versions.
Is there anything I might have missed with this suggestion? I'm happy to work on this if others agree on the general approach.
@lovell Interesting. Could you provide a few different examples on different prebuild-install commands and what the result would be? I'm not getting this fully.
If ARM binaries had been prebuilt with the proposed numeric flag using something like:
prebuild --arch-arm --arm-version=6 ...
prebuild --arch-arm --arm-version=7 ...to create tarballs such as:
module-v1.2.3-node-v64-linux-armv6.tar.gz
module-v1.2.3-node-v64-linux-armv7.tar.gz
then the install script would be updated to use the proposed boolean flag:
{
"scripts": {
"install": "prebuild-install --arm-version || node-gyp rebuild"
}
}and the relevant ARM version and therefore tarball would be determined via process.config.variables.arm_version.
Calling prebuild-install without the proposed flag would work as it currently does, in this example looking for module-v1.2.3-node-v64-linux-arm.tar.gz, so this suggestion is backwards compatible.
@lovell Makes total sense to me now. Just wanted to get the workflow/use-case into my head. Thanks!
Is there any progress on this?
@m4heshd Unlikely to get worked on. Several of us have moved on to prebuildify. Which can differentiate ARM versions. You might also like prebuildify-cross to cross-compile for ARM.
@vweevers Thank you for the quick response.
I've actually looked into both of those tools quite some time ago but unfortunately it's not affordable to ship 50+ prebuilt binaries in the package in my scenario. I still have to utilize prebuild-install to fetch the compatible binary. So building for armv6 and armv7 separately seems like a bust for me at least for now. 😟