Error when installing: `cannot find -lnlopt`

Question

Error when installing: `cannot find -lnlopt`

MiguelRodo opened this issue 2 years ago · comments

Hi

Thank you for all the effort maintaining this package.

So, I'm attempting to install nloptr on a CentOS 7 server to which I do not have root access. Simply running install.packages("nloptr") did not work as cmake's version is too low.

However, it seemed like from some comments on here and on Stackoverflow that it's a good idea to install nlopt "manually", and then try install nloptr in R.

So, I asked the system administrator to install nlopt, which was done (version 2.7.1). Since this wasn't installed in a standard path, I set PKG_CONFIG_PATH=/opt/exp_soft/nlopt-2.7.1/lib64/pkgconfig. However, running install.packages("nloptr") still throws an error. The primary error message seems to be this:

/usr/bin/ld: cannot find -lnlopt

I presume that this is some library. However, I'm not sure how to specify this. I tried using the --with... arguments to config.vars as specified in issue 33, but had no success and was told that these arguments are not recognised.

Here is the entire printed output from running install.packages("nloptr"):

* installing *source* package ‘nloptr’ ...
** package ‘nloptr’ successfully unpacked and MD5 sums checked
** using non-staged installation
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ -std=gnu++14 accepts -g... yes
checking how to run the C++ preprocessor... g++ -std=gnu++14 -E
checking whether we are using the GNU C++ compiler... (cached) yes
checking whether g++ -std=gnu++14 accepts -g... (cached) yes
checking for pkg-config... /opt/exp_soft/miniconda3-py39-usr/bin/pkg-config
checking if pkg-config knows NLopt... yes
checking for pkg-config checking NLopt version... >= 2.7.0
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gcc -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c init_nloptr.c -o init_nloptr.o
gcc -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c nloptr.c -o nloptr.o
g++ -std=gnu++11 -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c test-C-API.cpp -o test-C-API.o
g++ -std=gnu++11 -I"/opt/exp_soft/R-4.1.1/lib64/R/include" -DNDEBUG -I../inst/include -I/opt/exp_soft/nlopt-2.7.1/include -I'/home/rdxmig002/programs/r/lib/4.1/testthat/include' -I/usr/local/include   -fpic  -g -O2  -c test-runner.cpp -o test-runner.o
g++ -std=gnu++11 -shared -L/opt/exp_soft/R-4.1.1/lib64/R/lib -L/usr/local/lib64 -o nloptr.so init_nloptr.o nloptr.o test-C-API.o test-runner.o -lopenblas -lgfortran -lm -lquadmath -L/opt/exp_soft/nlopt-2.7.1/lib -lnlopt -L/opt/exp_soft/R-4.1.1/lib64/R/lib -lR
/usr/bin/ld: cannot find -lnlopt
collect2: error: ld returned 1 exit status
make: *** [nloptr.so] Error 1
ERROR: compilation failed for package ‘nloptr’
* removing ‘/home/rdxmig002/programs/r/lib/4.1/nloptr’
* restoring previous ‘/home/rdxmig002/programs/r/lib/4.1/nloptr’

Here is the output from sessionInfo():

R version 4.1.1 (2021-08-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /usr/lib64/libopenblas-r0.3.3.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] alwaysloaded_0.2.0

loaded via a namespace (and not attached):
[1] compiler_4.1.1 magrittr_2.0.2 tools_4.1.1

Dirk Eddelbuettel · Answer 1 · Tue Mar 15 2022 23:40:28 GMT+0800 (China Standard Time)

You are close, and you seem to diagnose this correctly. The automated process ask pkg-config about the location, it does not seem to get the right answer.

As a one-off, you could always a) remove script configure and b) hand-edit src/Makevars from src/Makevars.in to suit your location.

Dirk Eddelbuettel · Answer 2 · Tue Mar 15 2022 23:43:09 GMT+0800 (China Standard Time)

Sorry, I was wrong / too fast here. pkg-config finds it:

checking if pkg-config knows NLopt... yes
checking for pkg-config checking NLopt version... >= 2.7.0

What happens now looks like the dynamic (system-wide) linker not finding it. You can try setting LD_LIBRARY_PATH and alike; a higher-end solution on some systems is adding a file ending in .conf in the directory /etc/ld.so.conf.d/.

In simply terms, because we use dynamic linking, your nlopt installation needs to add its non-standard location to the linker's knowledge about paths to look in for it.

Miguel Rodo · Answer 3 · Tue Mar 15 2022 23:44:58 GMT+0800 (China Standard Time)

Great, thanks! I'm very new to these concepts, so I'll read up on your suggestions and revert (and thanks for the additional explanation).

Dirk Eddelbuettel · Answer 4 · Tue Mar 15 2022 23:46:52 GMT+0800 (China Standard Time)

It's more-or-less standard "non-standard" system administration. You made most of the right steps, happy to help for the last one.

Miguel Rodo · Answer 5 · Tue Mar 15 2022 23:48:29 GMT+0800 (China Standard Time)

Thanks! It's 6pm here in SA, so I'll have to get onto this tomorrow (you reply so quickly I felt I had to say that!:P thanks very much).

Miguel Rodo · Answer 6 · Wed Mar 16 2022 22:26:15 GMT+0800 (China Standard Time)

Hi again. So, I tried the tack of prepending two paths inside the nlopt folder to LD_LIBRARY_PATH. I added the following to .bashrc:

export LD_LIBRARY_PATH=/opt/exp_soft/nlopt-2.7.1/include:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/opt/exp_soft/nlopt-2.7.1/lib64:$LD_LIBRARY_PATH

I suppose that only the lib64 folder is required, so I tried with adding the include folder and without adding the include folder and got the same result.

I get the same error as before (I think exactly), specifically cannot find -lnlopt. But it seems like that would be inside lib64 of the nlopt-2.7.1 directory, as shown by ls /opt/exp_soft/nlopt-2.7.1/* yielding this:

/opt/exp_soft/nlopt-2.7.1/include:
nlopt.f  nlopt.h  nlopt.hpp

/opt/exp_soft/nlopt-2.7.1/lib64:
cmake  libnlopt.so  libnlopt.so.0  libnlopt.so.0.11.1  pkgconfig

/opt/exp_soft/nlopt-2.7.1/share:
man

I notice that the final line before the error message about not finding -libnlopt mentions /opt/exp_soft/nlopt-2.7.1/lib. This folder doesn't exist, though - it's lib64. I'm not sure if this is a problem, since I think the -L in front means this is a link and I don't think the link name has to match the target (reading up on this seems to say that the actual target is stored "in" the link as text).

Should I handle LD_LIBRARY_PATH differently, or try one of its "friends" or asking the sysadmin to add that .conf file?

Thanks again for the help.

Miguel Rodo · Answer 7 · Thu Mar 17 2022 13:51:08 GMT+0800 (China Standard Time)

So, our sysadmin created a symlink from lib to lib64. The package now installs and runs!

So, in conclusion for anyone with one or all of our problems, here are how they were solved (others should feel free to suggest edits):

cmake out of date:
- Install nlopt before attempting install of nloptr
pkg-config does not find nlopt due to non-standard install location:
- Add the following to to .bashrc:
  - export PKG_CONFIG_PATH=<path_to_nlopt>/<lib64_OR_lib>/pkgconfig
  - export LD_LIBRARY_PATH=<path_to_nlopt>/<lib64_OR_lib>:$LD_LIBRARY_PATH
  - Note that <lib64_OR_lib> means place lib64 or lib there (choose what appears in output of ls <path_to_nlopt>)
You have lib64 but not lib inside the nlopt folder, when nloptr assumes that there is a lib folder:
- Create a symlink from where the lib folder would be (namely <path_to_nlopt>/lib) to the lib64 folder you have (namely <path_to_nlopt>/lib64)
  - I didn't do this, but I suppose this means running the following command:
    - ln -s <path_to_nlopt>/lib64 <path_to_nlopt>/lib

I would close the issue, but I'll leave that to the package maintainer as the lib64 problem seems like a bug and this message might merit editing.

Dirk Eddelbuettel · Answer 8 · Thu Mar 17 2022 19:49:35 GMT+0800 (China Standard Time)

Thanks for working through this. The lib != lib64 issue seems related to the (upstream) NLopt project and its package config output on CentOS. The nloptr package is just a consumer of that information and trusts it (and you appear to have corrected an infelicity in the right place).

Miguel Rodo · Answer 9 · Thu Mar 17 2022 20:25:07 GMT+0800 (China Standard Time)

Oh, okay, right. I'll fine an issue on the stevengh/nlopt repo then, letting them know about it. Thanks for the LD_LIBRARY_PATH suggestion and explaining what's happened!