Nasty segfault with erroneous netcdf-4 data
durack1 opened this issue · comments
Describe the bug
I am able to reproduce a segfault when attempting to open a poorly formed netcdf-4 file. I would expect cdms to throw an OSError or similar error, rather than segfaulting
To Reproduce
Steps to reproduce the behavior:
The below is a grab of an ipython console output:
In [5]: getGlobalAtts(f)
getGlobalAtts: badFile: /p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc
Error: NetCDF: HDF error
ncopen: filename "/p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc": NetCDF: HDF error
Out[5]:
['/p/css03/esgf_publish/CMIP6/ScenarioMIP/CCCma/CanESM5/ssp126/r5i1p1f1/Omon/tauvo/gn/v20190306/tauvo_Omon_CanESM5_ssp126_r5i1p1f1_gn_201501-210012.nc',
OSError('NetCDF: HDF error')]
In [6]: getGlobalAtts(f)
Output from spyder call 'get_namespace_view':
*** Error in `~/anaconda3/envs/cdms315spy515cart020/bin/python': free(): invalid pointer: 0x00007fd568424bf0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x81329)[0x7fd570980329]
~anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5MM_xfree+0xf)[0x7fd4f9222dff]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x1aed3a)[0x7fd4f9251d3a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x1aeeb2)[0x7fd4f9251eb2]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5O_msg_reset+0x74)[0x7fd4f9252ca4]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__link_release_table+0x58)[0x7fd4f91db368]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__dense_iterate+0x14f)[0x7fd4f91d1ecf]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G__obj_iterate+0xd2)[0x7fd4f91e2b32]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5G_iterate+0x148)[0x7fd4f91d9b28]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(+0x16f5ab)[0x7fd4f92125ab]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../.././libhdf5.so.103(H5Literate+0xca)[0x7fd4f9213f4a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(+0x945de)[0x7fd5022b85de]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(+0x976ba)[0x7fd5022bb6ba]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(NC_open+0x238)[0x7fd502254618]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(nc_open+0x15)[0x7fd5022547b5]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/../../../libnetcdf.so.19(ncopen+0x24)[0x7fd502273b14]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0x7689)[0x7fd5023b3689]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0x931a)[0x7fd5023b531a]
~/anaconda3/envs/cdms315spy515cart020/lib/python3.9/site-packages/cdms2/Cdunif.cpython-39-x86_64-linux-gnu.so(+0xe67e)[0x7fd5023ba67e]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x15fd78)[0x558c65f90d78]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x316)[0x558c65f776a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x534c)[0x558c66015f2c]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1ac3c7)[0x558c65fdd3c7]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x3c2)[0x558c65f77752]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x4e0c)[0x558c660159ec]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x336)[0x558c65fd1366]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyObject_Call+0x106)[0x558c65f963a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyObject_MakeTpCall+0x316)[0x558c65f776a6]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x534c)[0x558c66015f2c]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x139900)[0x558c65f6a900]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x336)[0x558c65fd1366]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x1390d0)[0x558c65f6a0d0]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalCodeWithName+0x47)[0x558c6604f667]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyEval_EvalCodeEx+0x39)[0x558c6604f6a9]
~/anaconda3/envs/cdms315spy515cart020/bin/python(PyEval_EvalCode+0x1b)[0x558c6604f6cb]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x2930ad)[0x558c660c40ad]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x19972e)[0x558c65fca72e]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x5179)[0x558c66015d59]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x5179)[0x558c66015d59]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x167055)[0x558c65f98055]
~/anaconda3/envs/cdms315spy515cart020/bin/python(+0x19a579)[0x558c65fcb579]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x603)[0x558c660111e3]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyFunction_Vectorcall+0x19a)[0x558c65fd11ca]
~/anaconda3/envs/cdms315spy515cart020/bin/python(_PyEval_EvalFrameDefault+0x3b9)[0x558c66010f99]
Restarting kernel...Expected behavior
I would expect and OSError to be thrown, and no segfault
Screenshots or traceback
If applicable, add screenshots/traceback to help explain your problem.
Desktop (please complete the following information):
- OS: RHEL 7.9
- Browser NA
- Version cdms3.1.5
Environment Information
`conda info`
(cdms315spy515cart020) bash-4.2$ conda info
active environment : cdms315spy515cart020
active env location : ~/anaconda3/envs/cdms315spy515cart020
shell level : 3
user config file : ~/.condarc
populated config files :
conda version : 4.12.0
conda-build version : 3.21.4
python version : 3.8.8.final.0
virtual packages : __linux=3.10.0=0
__glibc=2.17=0
__unix=0=0
__archspec=1=x86_64
base environment : ~/anaconda3 (writable)
conda av data dir : ~/anaconda3/etc/conda
conda av metadata url : None
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : ~/anaconda3/pkgs
~/.conda/pkgs
envs directories : ~/anaconda3/envs
~/.conda/envs
platform : linux-64
user-agent : conda/4.12.0 requests/2.27.1 CPython/3.8.8 Linux/3.10.0-1160.42.2.el7.x86_64 rhel/7.9 glibc/2.17
UID:GID : 40336:1026
netrc file : None
offline mode : False
`conda config --show-sources`
`conda list --show-channel-urls`
Additional context
Add any other context about the problem here.