Cannot be installed under Docker Alpine
snics opened this issue · comments
I would like to use NodeGit under Docker with the Node alpine image but its doesn't work. Do you know one of the ways to solve the problem?
I install NodeGit with yarn i get the following error message:
Error: Error loading shared library /home/node/node_modules/nodegit/build/Release/nodegit.node: Exec format error
at Object.Module._extensions..node (module.js:598:18)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
at Module.require (module.js:513:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/node/node_modules/nodegit/dist/nodegit.js:11:12)
at Module._compile (module.js:569:30)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
at Module.require (module.js:513:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/home/node/src/utils/versioning.js:11:17)
at Module._compile (module.js:569:30)
at Object.Module._extensions..js (module.js:580:10)
at Module.load (module.js:503:32)
at tryModuleLoad (module.js:466:12)
at Function.Module._load (module.js:458:3)
at Module.require (module.js:513:17)
at require (internal/module.js:11:18)
I install NodeGit with npm i get the following error message:
> nodegit@0.20.1 install /home/node/node_modules/nodegit
> node lifecycleScripts/preinstall && node lifecycleScripts/install
[nodegit] Running pre-install script
[nodegit] Configuring libssh2.
{ Error: Command failed: /home/node/node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/home/node/node_modules/nodegit/vendor/openssl/openssl
/home/node/node_modules/nodegit/vendor/libssh2/missing: Unknown `--is-lightweight' option
Try `/home/node/node_modules/nodegit/vendor/libssh2/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing
/home/node/node_modules/nodegit/vendor/libssh2/configure: ./configure.lineno: line 1: /usr/bin/file: not found
configure: error: No crypto library found!
Try --with-libssl-prefix=PATH
or --with-libgcrypt-prefix=PATH
or --with-wincng on Windows
at ChildProcess.exithandler (child_process.js:252:12)
at emitTwo (events.js:125:13)
at ChildProcess.emit (events.js:213:7)
at maybeClose (internal/child_process.js:887:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:208:5)
killed: false,
code: 1,
signal: null,
cmd: '/home/node/node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/home/node/node_modules/nodegit/vendor/openssl/openssl' }
/home/node/node_modules/nodegit/vendor/libssh2/missing: Unknown `--is-lightweight' option
Try `/home/node/node_modules/nodegit/vendor/libssh2/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing
/home/node/node_modules/nodegit/vendor/libssh2/configure: ./configure.lineno: line 1: /usr/bin/file: not found
configure: error: No crypto library found!
Try --with-libssl-prefix=PATH
or --with-libgcrypt-prefix=PATH
or --with-wincng on Windows
[nodegit] ERROR - Could not finish preinstall
{ Error: Command failed: /home/node/node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/home/node/node_modules/nodegit/vendor/openssl/openssl
/home/node/node_modules/nodegit/vendor/libssh2/missing: Unknown `--is-lightweight' option
Try `/home/node/node_modules/nodegit/vendor/libssh2/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing
/home/node/node_modules/nodegit/vendor/libssh2/configure: ./configure.lineno: line 1: /usr/bin/file: not found
configure: error: No crypto library found!
Try --with-libssl-prefix=PATH
or --with-libgcrypt-prefix=PATH
or --with-wincng on Windows
at ChildProcess.exithandler (child_process.js:252:12)
at emitTwo (events.js:125:13)
at ChildProcess.emit (events.js:213:7)
at maybeClose (internal/child_process.js:887:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:208:5)
killed: false,
code: 1,
signal: null,
cmd: '/home/node/node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/home/node/node_modules/nodegit/vendor/openssl/openssl' }
npm info lifecycle nodegit@0.20.1~install: Failed to exec install script
npm WARN vao-api@1.0.0 No repository field.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! nodegit@0.20.1 install: `node lifecycleScripts/preinstall && node lifecycleScripts/install`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the nodegit@0.20.1 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2017-08-30T10_48_59_977Z-debug.log
I did some testing myself but got stuck at libcurl-gnutls.so.4
. I couldn't figure out what package would add that shared library to Alpine Linux.
> nodegit@0.20.1 postinstall /node_modules/nodegit
> node lifecycleScripts/postinstall
[nodegit] WARN - Could not finish postinstall
{ Error: Command failed: node "/node_modules/nodegit/dist/nodegit.js"
/node_modules/nodegit/dist/nodegit.js:15
throw ex;
^
Error: Error loading shared library libcurl-gnutls.so.4: No such file or directory (needed by /node_modules/nodegit/build/Release/nodegit.node)
at Object.Module._extensions..node (module.js:602:18)
at Module.load (module.js:507:32)
at tryModuleLoad (module.js:470:12)
at Function.Module._load (module.js:462:3)
at Module.require (module.js:517:17)
at require (internal/module.js:11:18)
at Object.<anonymous> (/node_modules/nodegit/dist/nodegit.js:11:12)
at Module._compile (module.js:573:30)
at Object.Module._extensions..js (module.js:584:10)
at Module.load (module.js:507:32)
at ChildProcess.exithandler (child_process.js:270:12)
at emitTwo (events.js:125:13)
at ChildProcess.emit (events.js:213:7)
at maybeClose (internal/child_process.js:927:16)
at Process.ChildProcess._handle.onexit (internal/child_process.js:211:5)
killed: false,
code: 1,
signal: null,
cmd: 'node "/node_modules/nodegit/dist/nodegit.js"' }
@rcjsuen I have installed the following package libgit2-dev
and I think this has all dependencies for NodeGit. For example I add my dockerfile below
FROM node:8.1.0-alpine
COPY ./ /home/node
WORKDIR /home/node
# Git install
RUN mkdir /nodegit && \
cd /nodegit && \
apk update && \
apk upgrade && \
apk add git libgit2-dev && \
apk add python tzdata pkgconfig build-base && \
yarn add -E nodegit@0.20 && \
apk del python tzdata pkgconfig build-base && \
rm yarn.lock package.json && \
rm -rf /tmp/* /var/cache/apk/* && \
yarn cache clean && \
# Install Node modules
RUN rm -rf package-lock.json node_modules \
&& apk add --update --no-cache --virtual .build-deps \
curl \
g++ \
gcc \
gnupg \
libgcc \
make \
alpine-sdk \
python \
&& npm config set unsafe-perm true \
&& npm install \
&& npm prune --production \
&& apk del .build-deps \
&& rm -rf /usr/share/man /tmp/* /var/cache/apk/* \
&& ls -a /nodegit
EXPOSE 8000
USER node
CMD ["node", "index"]
I think this is a duplicate of #1246.
There are two ways to solve the problem.
-
Create the missing symlink that the precompiled binary is depending on.
sudo ln -s /usr/lib64/libcurl.so.4 /usr/lib64/libcurl-gnutls.so.4
-
Force nodegit to be recompiled at install time.
BUILD_ONLY=true npm install nodegit
The second option slows down installation considerably.
I couldn't figure out what package would add that shared library to Alpine Linux.
The problem is, this is not the normal name of this library. It's normally called libcurl.so.4 and installed by libcurl.
I'm trying to run nodegit in a Docker container using the base image node:8-alpine
but having no luck with the error:
RUN npm install nodegit
---> Running in b99980a70b1b
> nodegit@0.20.3 install /node_modules/nodegit
> node lifecycleScripts/preinstall && node lifecycleScripts/install
[nodegit] Running pre-install script
[nodegit] Configuring libssh2.
{ Error: Command failed: /node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/node_modules/nodegit/vendor/openssl/openssl
/node_modules/nodegit/vendor/libssh2/missing: Unknown `--is-lightweight' option
Try `/node_modules/nodegit/vendor/libssh2/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing
configure: error: in `/node_modules/nodegit/vendor/libssh2':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
at ChildProcess.exithandler (child_process.js:275:12)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:925:16)
at Socket.stream.socket.on (internal/child_process.js:346:11)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at Pipe._handle.close [as _onclose] (net.js:554:12)
killed: false,
code: 1,
signal: null,
cmd: '/node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/node_modules/nodegit/vendor/openssl/openssl' }
/node_modules/nodegit/vendor/libssh2/missing: Unknown `--is-lightweight' option
Try `/node_modules/nodegit/vendor/libssh2/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing
configure: error: in `/node_modules/nodegit/vendor/libssh2':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
[nodegit] ERROR - Could not finish preinstall
{ Error: Command failed: /node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/node_modules/nodegit/vendor/openssl/openssl
/node_modules/nodegit/vendor/libssh2/missing: Unknown `--is-lightweight' option
Try `/node_modules/nodegit/vendor/libssh2/missing --help' for more information
configure: WARNING: 'missing' script is too old or missing
configure: error: in `/node_modules/nodegit/vendor/libssh2':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
at ChildProcess.exithandler (child_process.js:275:12)
at emitTwo (events.js:126:13)
at ChildProcess.emit (events.js:214:7)
at maybeClose (internal/child_process.js:925:16)
at Socket.stream.socket.on (internal/child_process.js:346:11)
at emitOne (events.js:116:13)
at Socket.emit (events.js:211:7)
at Pipe._handle.close [as _onclose] (net.js:554:12)
killed: false,
code: 1,
signal: null,
cmd: '/node_modules/nodegit/vendor/libssh2/configure --with-libssl-prefix=/node_modules/nodegit/vendor/openssl/openssl' }
npm WARN microclimate-portal@1.0.0 No description
npm WARN microclimate-portal@1.0.0 No repository field.
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! nodegit@0.20.3 install: `node lifecycleScripts/preinstall && node lifecycleScripts/install`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the nodegit@0.20.3 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-02-20T13_59_09_275Z-debug.log
The command '/bin/sh -c npm install nodegit --production' returned a non-zero code: 1
I've tried both the suggested options above and installing via a git clone but it still doesn't work.
Any suggestions of things I can try? apk packages to add?
Thanks in advance
In order to install on Alpine, you need the following packages:
apk --no-cache -q add build-base libgit2-dev
You also need this symlink:
ln -s /usr/lib/libcurl.so.4 /usr/lib/libcurl-gnutls.so.4
Then you can install and use nodegit on Alpine.
@mojavelinux Thanks for getting back to me.
The above allows me to run nodegit but when I Git.clone
a valid repository it crashes my application with a segmentation error.
I've not seen anything to do with segmentation errors in alpine in an issue so is this new or is there already a known fix?
Segmentation fault
npm ERR! code ELIFECYCLE
npm ERR! errno 139
npm ERR! github-api@1.0.0 start: `node server.js`
npm ERR! Exit status 139
npm ERR!
npm ERR! Failed at the github-api@1.0.0 start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
npm ERR! A complete log of this run can be found in:
npm ERR! /root/.npm/_logs/2018-02-21T09_59_06_487Z-debug.log
I'm using nodegit successfully in my application on Alpine, so I know it works in general (including clone). However, what I've learned is that if you forget to await an operation and close the repository prematurally, then nodegit will hard crash. So if not called correctly, nodegit is known to segfault in my experience.
What I recommend is to create a minimal script that reproduces the problem. That will either allow you to pinpoint the problem or identify a problem in nodegit.
@mojavelinux this is all I am doing to cause it to crash.
In my case I don't use the repository object. All I want to do is clone a repository and then send a 200 status back in the response from my API.
You've mentioned using await but I'm not sure where/why I'd add that in
const Git = require("nodegit");
const express = require('express');
const bodyParser = require('body-parser');
const app = express();
const serverPort = 9030;
const server = app.listen(serverPort);
const path = require('path');
app.use(bodyParser.urlencoded({ extended: false }));
app.post('/api/v1/import', function(req, res) {
if (!req.body.repo) res.sendStatus(400).end();
try {
let repo = req.body.repo;
// get the name of the git repository from the repo given
let dir = path.basename(repo.replace('.git', ''));
if (req.body.dir) dir = req.body.dir;
// make dir the full path
dir = `${__dirname}${dir}`;
console.log(repo);
console.log(dir);
Git.Clone(repo, dir).then(function() {
res.sendStatus(200).end();
}).catch(function(err) {
if (err.message.indexOf('authentication required') > -1) {
res.header('authentication-required', 'true');
}
res.sendStatus(500).end();
console.log(err);
});
} catch (err) {
res.sendStatus(500).end();
console.log(err);
}
});
Here's how I use it. Notice I can't clone an https repo without providing fetch options.
Notice I'm using Git.Clone.clone, not Git.Clone.
I have over 100 tests cloning repositories of different types, so I'm pretty confident that call to clone is a good one.
@mojavelinux I've added the Git.Clone.clone and added your fetchOpts and its function as a test case, still failing with a segmentation error.
I've simplified my function to be:
function checkWorks() {
// Uses the gitFetchOptions from your gitlab
Git.Clone.clone('https://github.com/nodegit/nodegit.git', '/tmp/nodegit', { bare: 1, fetchOpts: getFetchOptions() }).then(function(repo) {
console.log('done');
}).catch(function(err) {
console.log(err);
});
}
But I'm still getting the same segmentation error. At this point i'm clueless on what I can do to fix it. Should I be adding an await somewhere or a repo.close()?
Thanks again
This is my Dockerfile in case I've created the issue in it
FROM node:8-alpine
RUN apk --update --no-cache -q add docker zip git grep bash sudo curl build-base libgit2-dev
RUN ln -s /usr/lib/libcurl.so.4 /usr/lib/libcurl-gnutls.so.4
ADD package.json server.js ./
RUN npm install
CMD ["npm", "start"]
You are indeed correct. Cloning a remote repository on Alpine fails in a segfault.
I distilled the script even further:
const git = require('nodegit')
;(async () => {
const repo = await git.Clone.clone('http://github.com/sindresorhus/awesome.git', 'clone', {
fetchOpts: {
callbacks: {
certificateCheck: () => 1,
},
},
})
})()
You may be interested in researching isomorphic git (isogit for short). It's a git client written completely in JavaScript. It's quite as fast as nodegit, but it is a heck of a lot more stable and easy to install. I'm evaluating it as a substitute git client in my application.
I now have a deeper understanding of this issue and have discovered a workaround.
To install the pre-built nodegit package on Alpine, the following packages are required:
apk --no-cache add gcc libc-dev libressl-dev
That allows you to install nodegit and use it to read and write local repositories. However, if you attempt to clone a remote repository, the process will segfault. The workaround for this problem is to rebuild nodegit at the time it is installed.
To rebuild nodegit at installation time, you first need to following additional packages:
apk --no-cache add make python curl-dev g++
Next, install nodegit with the BUILD_ONLY=true
environment variable:
BUILD_ONLY=true npm install nodegit
or, with yarn:
BUILD_ONLY=true yarn add nodegit
The BUILD_ONLY=true
environment variable can be used when installing any package which itself depends on nodegit. It just needs to be set at the time nodegit is being installed.
Compilation takes a long time, but I can confirm that it does solve the segfault problem. I tested this process using the node:8-alpine
Docker image.
I've distilled these steps in the following comment: https://gitlab.com/antora/docker-antora/issues/2#note_61275108
Thanks for sharing your workaround folks! Here's my solution for a microservice on node 10:
FROM node:10.15.1-alpine
RUN mkdir -p /app/
WORKDIR /app/
COPY package.json yarn.lock /app/
## https://github.com/nodegit/nodegit/issues/1361
RUN apk --no-cache add curl-dev g++ make python \
&& BUILD_ONLY=true YARN_CACHE_FOLDER=/dev/shm/yarn_cache yarn --production \
&& apk del g++ make python
COPY src /app/src/
CMD ["yarn", "start:prod"]
Note that I install and delete packages in one RUN
. This makes the final image significantly smaller since only curl-dev
makes it into it.
Hi @mojavelinux Hi have today experienced issues with nodegit inside docker container node:8-alpine
, pretty much the same as James
So I just added the commands you suggested :
In order to install on Alpine, you need the following packages:
apk --no-cache -q add build-base libgit2-dev
You also need this symlink:
ln -s /usr/lib/libcurl.so.4 /usr/lib/libcurl-gnutls.so.4
Then you can install and use nodegit on Alpine.
I still get errors, which you can reproduce with a tag I made for you :
https://github.com/Jean-Baptiste-Lasselle/provision-mjml-server/releases/tag/MOJAVELINUX_REFERENCE
To run the test, just execute :
docker-compose down --rmi all && git pull && docker-compose down --rmi all && docker-compose up -d --force-recreate
Thank you all team for leaving so many comments, working on issues, feels great to feel support (and I'm dying to get my new MJML web editor ... :) )
@mojavelinux Thanks, this worked for us, too.
However, the libgit2 compilation indeed takes some time (~3 minutes for us), so we decided to switch to the "slim" variant of the official debian node image (node:10-stretch-slim). There, you can just apt install libcurl3-gnutls
to get libcurl-gnutls.so.4
.
As for me, it worked fine on alpine 3.8 but not with node:10-alpine3.9
it faults Error loading shared library /usr/src/Hilary/node_modules/nodegit/build/Release/nodegit.node: Exec format error
Any ideas?
@rcjsuen I have installed the following package
libgit2-dev
and I think this has all dependencies for NodeGit. For example I add my dockerfile belowFROM node:8.1.0-alpine COPY ./ /home/node WORKDIR /home/node # Git install RUN mkdir /nodegit && \ cd /nodegit && \ apk update && \ apk upgrade && \ apk add git libgit2-dev && \ apk add python tzdata pkgconfig build-base && \ yarn add -E nodegit@0.20 && \ apk del python tzdata pkgconfig build-base && \ rm yarn.lock package.json && \ rm -rf /tmp/* /var/cache/apk/* && \ yarn cache clean && \ # Install Node modules RUN rm -rf package-lock.json node_modules \ && apk add --update --no-cache --virtual .build-deps \ curl \ g++ \ gcc \ gnupg \ libgcc \ make \ alpine-sdk \ python \ && npm config set unsafe-perm true \ && npm install \ && npm prune --production \ && apk del .build-deps \ && rm -rf /usr/share/man /tmp/* /var/cache/apk/* \ && ls -a /nodegit EXPOSE 8000 USER node CMD ["node", "index"]
is this works ?
I have some error like this:
Error loading shared library libgssapi_krb5.so.2: No such file or directory
using multi stage dockerfile. Should I install krb5-libs
or other libs in the last stage ?
@prakasa-tkpd +1 same problem
RUN apk --no-cache add krb5 pcre
it's work for me @prakasa-tkpd
Has anyone gotten this to work on node:12-alpine
? The rebuild solution worked for me with alpine 3.8, but not 3.9.
FROM node:12-alpine
RUN apk --update --no-cache -q add libgit2-dev
ADD package.json index.js ./
RUN apk --no-cache --virtual .node-gyp-compilation-dependencies add \
g++ \
make \
python \
curl-dev \
&& apk --no-cache add \
bash \
ca-certificates \
krb5-dev \
libgit2-dev
RUN BUILD_ONLY=true npm install
RUN apk del .node-gyp-compilation-dependencies
CMD ["node", "index.js"]
const nodegit = require('nodegit');
nodegit.Clone.clone('https://github.com/nodegit/nodegit/', 'test-test', {})
.then(() => console.log('all good!'))
.catch(e => console.log('error!', e));
{
"name": "test",
"version": "1.0.0",
"main": "index.js",
"dependencies": {
"nodegit": "0.27.0"
}
}
> docker build -t tester .
> docker run tester
Segmentation fault
@pheel Im having issues dockerizing nodegit 0.27.0 as well. Were you able to get this working?
@Perronef5 low-hanging solution for us was to use node:12-buster-slim
as a base instead. Works out of the box with npm's compiled nodegit. Just make sure to RUN apt-get update && apt-get install -y libgssapi-krb5-2
in the build.
@pheel That works! Thanks 🔥
@pheel Could you share the whole Dockerfile
with the node:12-buster-slim
approach? I can't seem to make it work by just adding that line so probably I am missing something.
For others that face this issue, I was able to address it using #1361 (comment).
FROM node:14-buster-slim
RUN apt-get update && apt-get install -y libgssapi-krb5-2
...