[5.5.3]: java.io.EOFException while reading some BUFR files (solution: just update the bufr tables)
JP95Git opened this issue · comments
Versions impacted by the bug
v5.x
What went wrong?
Hi,
I am unable to read BUFR files from these stations:
- SOWR = Wroclaw (PL) or Leba (PL)
- LFPW = Trappes (FR)
The files from SOWR are generated by MODEM: https://www.meteomodem.com/
An example file is attached: Z__C_SOWR_20230716010100_aero1,IUSD02_SOWR_160000____RSSO.bin
I was able to read this file with this online tool https://kunden.dwd.de/bufrviewer/ and with ecCodes ( https://confluence.ecmwf.int/display/ECC ). I Also attached a dump of this file as bufr_dump.dat generated by bufr_dump from ecCodes.
Stacktrace after opening the file in toolsUI 5.5.3 and click on "Show Data" is attached in "Relevant stack trace".
I get the same stack trace in toolsUI 5.5.3, 5.5.2, 5.4.1, 5.3.3.
While using NetCDF Java 5.3.3 in my own program I got a similar stack trace:
java.io.EOFException: null
at ucar.nc2.iosp.BitReader.nextByte(BitReader.java:163)
at ucar.nc2.iosp.BitReader.bits2UInt(BitReader.java:104)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readNumericData(MessageUncompressedDataReader.java:267)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readData(MessageUncompressedDataReader.java:244)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.makeArraySequenceUncompressed(MessageUncompressedDataReader.java:364)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readData(MessageUncompressedDataReader.java:203)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readData(MessageUncompressedDataReader.java:132)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readEntireMessage(MessageUncompressedDataReader.java:88)
at ucar.nc2.iosp.bufr.BufrIosp2$SeqIter.readMessage(BufrIosp2.java:266)
at ucar.nc2.iosp.bufr.BufrIosp2$SeqIter.readNextMessage(BufrIosp2.java:255)
at ucar.nc2.iosp.bufr.BufrIosp2$SeqIter.hasNext(BufrIosp2.java:211)
at ucar.ma2.ArraySequence.extractMemberArrayFromIteration(ArraySequence.java:319)
at ucar.ma2.ArraySequence.extractMemberArray(ArraySequence.java:88)
at ucar.nc2.Variable.reallyRead(Variable.java:757)
at ucar.nc2.Variable._read(Variable.java:731)
at ucar.nc2.Variable.read(Variable.java:611)
at ucar.nc2.dataset.VariableDS.reallyRead(VariableDS.java:437)
at ucar.nc2.dataset.VariableDS._read(VariableDS.java:426)
at ucar.nc2.Variable.read(Variable.java:611)
at ucar.nc2.Variable.getScalarData(Variable.java:708)
at ucar.nc2.Variable.readScalarString(Variable.java:697)
-> My own code is shown here.I reported this issue last year via email and got this response:
The netcdf-java library is not particularly robust about handling bufr files, and is very particular about how the bufr table needs to be formatted. An EOFException usually means that's something's not quite right with the bufr table. Unfortunately, Unidata doesn't currently have the resources to work on the bufr module, so it will probably be a while before we're able to investigate why this error occurs with some files. In the meantime, this is the information we have available on how the netcdf-java library handles bufr tables: https://docs.unidata.ucar.edu/netcdf-java/5.5/userguide/bufr_tables.html
Relevant stack trace
java.io.EOFException
at ucar.nc2.iosp.BitReader.nextByte(BitReader.java:163)
at ucar.nc2.iosp.BitReader.bits2UInt(BitReader.java:104)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readNumericData(MessageUncompressedDataReader.java:267)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readData(MessageUncompressedDataReader.java:244)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.makeArraySequenceUncompressed(MessageUncompressedDataReader.java:364)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readData(MessageUncompressedDataReader.java:203)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readData(MessageUncompressedDataReader.java:132)
at ucar.nc2.iosp.bufr.MessageUncompressedDataReader.readEntireMessage(MessageUncompressedDataReader.java:88)
at ucar.nc2.iosp.bufr.BufrIosp2$SeqIter.readMessage(BufrIosp2.java:266)
at ucar.nc2.iosp.bufr.BufrIosp2$SeqIter.readNextMessage(BufrIosp2.java:255)
at ucar.nc2.iosp.bufr.BufrIosp2$SeqIter.hasNext(BufrIosp2.java:211)
at ucar.ma2.ArraySequence.extractMemberArrayFromIteration(ArraySequence.java:315)
at ucar.ma2.ArraySequence.extractMemberArray(ArraySequence.java:83)
at ucar.ma2.ArrayStructureMA.factoryMA(ArrayStructureMA.java:224)
at ucar.nc2.dataset.StructureDS.convert(StructureDS.java:284)
at ucar.nc2.dataset.SequenceDS.read(SequenceDS.java:89)
at ucar.nc2.ui.op.DatasetViewer.showData(DatasetViewer.java:634)
at ucar.nc2.ui.op.DatasetViewer.access$1200(DatasetViewer.java:56)
at ucar.nc2.ui.op.DatasetViewer$NestedTable$3.actionPerformed(DatasetViewer.java:477)
at javax.swing.AbstractButton.fireActionPerformed(Unknown Source)
at javax.swing.AbstractButton$Handler.actionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.fireActionPerformed(Unknown Source)
at javax.swing.DefaultButtonModel.setPressed(Unknown Source)
at javax.swing.AbstractButton.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI.doClick(Unknown Source)
at javax.swing.plaf.basic.BasicMenuItemUI$Handler.mouseReleased(Unknown Source)
at java.awt.Component.processMouseEvent(Unknown Source)
at javax.swing.JComponent.processMouseEvent(Unknown Source)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEventImpl(Unknown Source)
at java.awt.EventQueue.access$500(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.awt.EventQueue$3.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue$4.run(Unknown Source)
at java.awt.EventQueue$4.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)Relevant log messages
No response
If you have an example file that you can share, please attach it to this issue.
Attached is bufr.zip which contains the BUFR file Z__C_SOWR_20230716010100_aero1,IUSD02_SOWR_160000____RSSO.bin and the dump bufr_dump.dat.
bufr.zip
Code of Conduct
- I agree to follow the UCAR/Unidata Code of Conduct
I was able to read the file by using the latest (version 40) of the BUFR tables from
https://community.wmo.int/en/activity-areas/wis/latest-version
BufrTables.addLookupFile("path/to/bufr/bufr-tables.csv");
path/to/bufr/bufr-tables.csv
WMO.any, 0, 0, -1, 0, -1, path/to/bufr/BUFRCREX_TableB_en.xml, wmo_xml , path/to/bufr/BUFR_TableD_en.xml, wmo_xml , localWmo
Works with netcdf-java 5.5.3 and netcdf-java 5.3.3
@haileyajohnson
Because the stack trace was very misleading you should update the BUFR tables in netcdf-java, or else other users may encounter the same problem. Thanks.