4.9.2 build fails without --disable-shared

Question

4.9.2 build fails without --disable-shared

burlen opened this issue 8 months ago · comments

the version of the software with which you are encountering an issue
netcdf 4.9.2
environmental information (i.e. Operating System, compiler info, java version, python version, etc.)
Fedora 35, gcc 12.2.1
a description of the issue with the steps needed to reproduce it
The build fails with many undefined symbols during linking.eg.

/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_open'

note: If I add --disable-shared the build succeeds.

steps to reproduce

$./configure CC=h5pcc --prefix=`pwd`/../netcdf-c-4.9.2-install --disable-dap --enable-netcdf-4 --enable-parallel4
$ make

I could not upload the output of the build because of the max 65536 character limit.

Ward Fisher · Answer 1 · Sat Oct 14 2023 07:15:47 GMT+0800 (China Standard Time)

Can you attach the config.log file generated when you configure without --disable-shared? I'm having an issue reproducing this, and want to get an idea of what's going on under the hood. Thanks!

Burlen Loring · Answer 2 · Tue Oct 17 2023 00:27:52 GMT+0800 (China Standard Time)

config.log

and here's the output from the command that failed:

Making all in ncgen3
make[2]: Entering directory '/home/bloring/work/teca/netcdf-c-4.9.2/ncgen3'
/bin/sh ../libtool  --tag=CC   --mode=link h5pcc  -fno-strict-aliasing    -o ncgen3 main.o load.o escapes.o getfill.o init.o genlib.o ncgeny.o ../liblib/libnetcdf.la -lm -lz -lsz -lbz2 -lzstd -lxml2 -lcurl 
libtool: link: h5pcc -fno-strict-aliasing -o .libs/ncgen3 main.o load.o escapes.o getfill.o init.o genlib.o ncgeny.o  ../liblib/.libs/libnetcdf.so -lm -lz -lsz -lbz2 -lzstd -lxml2 -lcurl -Wl,-rpath -Wl,/home/bloring/work/teca/netcdf-c-4.9.2/../netcdf-c-4.9.2-install/lib
/bin/ld: load.o: in function `load_netcdf':
load.c:(.text+0x1b02): undefined reference to `nc_put_vara_schar'
/bin/ld: load.c:(.text+0x1b32): undefined reference to `nc_put_vara_text'
/bin/ld: load.c:(.text+0x1b62): undefined reference to `nc_put_vara_short'
/bin/ld: load.c:(.text+0x1b92): undefined reference to `nc_put_vara_int'
/bin/ld: load.c:(.text+0x1bbf): undefined reference to `nc_put_vara_float'
/bin/ld: load.c:(.text+0x1bec): undefined reference to `nc_put_vara_double'
/bin/ld: genlib.o: in function `gen_netcdf':
genlib.c:(.text+0x21): undefined reference to `nc_create'
/bin/ld: genlib.c:(.text+0x8f): undefined reference to `nc_def_dim'
/bin/ld: genlib.c:(.text+0x12b): undefined reference to `nc_def_var'
/bin/ld: genlib.c:(.text+0x247): undefined reference to `nc_put_att_schar'
/bin/ld: genlib.c:(.text+0x2c5): undefined reference to `nc_put_att_text'
/bin/ld: genlib.c:(.text+0x366): undefined reference to `nc_put_att_short'
/bin/ld: genlib.c:(.text+0x407): undefined reference to `nc_put_att_int'
/bin/ld: genlib.c:(.text+0x4a8): undefined reference to `nc_put_att_float'
/bin/ld: genlib.c:(.text+0x549): undefined reference to `nc_put_att_double'
/bin/ld: genlib.c:(.text+0x593): undefined reference to `nc_set_fill'
/bin/ld: genlib.c:(.text+0x5ad): undefined reference to `nc_enddef'
/bin/ld: genlib.o: in function `cl_netcdf':
genlib.c:(.text+0x3a59): undefined reference to `nc_close'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_hashmapget'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_copy_data'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJcvt'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_insert_enum'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_put_att'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcacheremove'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_hashmapadd'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_copy_data_all'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcachekey'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_inq_compound_fieldindex'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOOP_inq_var_filter_info'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOOP_inq_var_filter_ids'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_inq_enum_member'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_def_vlen'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_get_att'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_def_enum'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCDEFAULT_get_vars'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_http_read'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `printhashmap'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcachenew'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJinsert'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOOP_inq_filter_avail'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJnewstringn'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJnew'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJdictget'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_inq_enum_ident'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_insert_array_compound'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_hashmapfree'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_inq_user_type'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_hashmapnew'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJparse'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcachetouch'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCDEFAULT_put_vars'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcachelast'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcachefree'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `move_in_NCList'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_http_size'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_hashmapremove'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_http_close'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_get_vlen_element'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJaddstring'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJreclaim'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJnewstring'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc__pseudofd'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJunparse'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_inq_grpname_full'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_put_vlen_element'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcachelookup'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_free_vlen'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_def_compound'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_inq_var_quantize'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_def_opaque'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_check_id'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_atomictypename'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_def_var_filter'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_reclaim_data'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCDEFAULT_put_varm'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_reclaim_data_all'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_insert_array_compound'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJparsen'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_inq_compound_field'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCDEFAULT_get_varm'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NCJappend'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `ncxcacheinsert'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_hashmapdeactivate'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_open'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nctypelen'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_atomictypelen'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_def_var_quantize'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_inq_var_filter_info'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `nc_http_init'
/bin/ld: ../liblib/.libs/libnetcdf.so: undefined reference to `NC_NOTNC4_insert_compound'
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:771: ncgen3] Error 1
make[2]: Leaving directory '/home/bloring/work/teca/netcdf-c-4.9.2/ncgen3'
make[1]: *** [Makefile:769: all-recursive] Error 1
make[1]: Leaving directory '/home/bloring/work/teca/netcdf-c-4.9.2'
make: *** [Makefile:614: all] Error 2

Ward Fisher · Answer 3 · Wed Oct 18 2023 05:04:17 GMT+0800 (China Standard Time)

Thank you, this is a peculiar issue. What version of libhdf5 are you building against, out of curiosity? I'm curious about the output of h5pcc -showconfig, and also the libnetcdf.settings file generated after you run configure. This might help me recreate the issue, which I'm struggling to do thus far. Thanks!

Ward Fisher · Answer 4 · Wed Oct 18 2023 05:36:40 GMT+0800 (China Standard Time)

Also, what happens if you try setting CC=mpicc instead of h5pcc?

Burlen Loring · Answer 5 · Thu Oct 19 2023 01:16:52 GMT+0800 (China Standard Time)

Your settings don't all for the upload of this file. "We don't support that file type. Try again" is the message.

Burlen Loring · Answer 6 · Thu Oct 19 2023 01:19:41 GMT+0800 (China Standard Time)

HDF5 Version: 1.12.1, installed via package manager, from Fedora 37 (I had mentioned 35 above, but turns out it's 37)

Ryan May · Answer 7 · Thu Oct 19 2023 01:20:57 GMT+0800 (China Standard Time)

Your settings don't all for the upload of this file. "We don't support that file type. Try again" is the message.

Unfortunately, that's a GitHub issue. It should work if you zip the file before uploading.

Burlen Loring · Answer 8 · Thu Oct 19 2023 01:25:55 GMT+0800 (China Standard Time)

Also, what happens if you try setting CC=mpicc instead of h5pcc?

That solved the issue. And netcdf found hdf5 on its own. No additional configure options were needed.

Thank you for the help!

Ward Fisher · Answer 9 · Thu Oct 19 2023 01:27:03 GMT+0800 (China Standard Time)

Thanks! Interesting, I wasn't very familiar with h5pcc, and I was able to recreate the issue you were seeing, but invoking mpicc directly worked. I'm going to poke around, I have a meeting scheduled with the HDF dev team, maybe we can see why this happened, and what would be needed to fix it. Thanks @burlen !