Won't parse N1695761485_1_CALIB.LBL
rbeyer opened this issue · comments
Hm, 1.0.0-alpha.2 does not parse this:
% pvl_validate -v N1695761485_1_CALIB.LBL
ERROR: PDS3 load error /Users/rbeyer/N1695761485_1_CALIB.LBL (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "PRODUCT_ID" : line 78 column 1 (char 2492)')
ERROR: ODL load error /Users/rbeyer/N1695761485_1_CALIB.LBL (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "PRODUCT_ID" : line 78 column 1 (char 2492)')
ERROR: PVL load error /Users/rbeyer/N1695761485_1_CALIB.LBL (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "PRODUCT_ID" : line 78 column 1 (char 2492)')
ERROR: ISIS load error /Users/rbeyer/N1695761485_1_CALIB.LBL (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "PRODUCT_ID" : line 78 column 1 (char 2492)')
ERROR: Omni load error /Users/rbeyer/N1695761485_1_CALIB.LBL (LexerError(...), 'Expecting an Aggregation Block, an Assignment Statement, or an End Statement, but found "PRODUCT_ID" : line 78 column 1 (char 2492)')
PDS3 | does NOT load |
ODL | does NOT load |
PVL | does NOT load |
ISIS | does NOT load |
Omni | does NOT load |
I see a potential issue on line 77: PRODUCT_CREATION_TIME = 2012-12- 5T20:44:07 whitespace in a date
Ha! I didn't even look at the spot that pvl_validate pointed to, as I was (am) working on something else. So that's not a problem with the parser, that's a malformed label (yay, go new parser, although the error message isn't super helpful). That means that's not an issue on our library, but an Issue that needs to be filed with the PDS, because they have bad data.
I can confirm that removing that space allows pvl_validate to run clean. Also, I can validate that a key can only have a single value, and the bare space (since this 'date' is not enclosed in quotes) means that there are two 'values' for this 'parameter' which is bogus according to the PVL specs.
There's hundreds more... let's see if they appear in the INDEX file like that, then I can scan where it starts. So far, anecdotal evidences shows this happens whenever data was produced when the time was at single digit hours.
Should the response be to kick the issue back to PDS to fix or to include logic / error checking that works around this issue in PVL (w/ an appropriate warning)? The chance of the PDS data being fixed for non-active missions is probably low.
I think in any case, the correct response is to notify the PDS of this bad data. It explicitly breaks their own specifications, and I'm sure they want to know about it. Whether they can or will do anything about it is unknown.
We could certainly attempt to put in some logic that would attempt to identify a date string with a space inserted into it. We have other existing accommodations in the OmniParser to deal with typical spec-breaking behavior, and also expose that capability to users that can define their own descendent of OmniParser where they can override parse_module_post_hook() or parse_value_post_hook() to deal with these kinds of situations if the codebase doesn't already.
So I guess the question is: how pervasive is this?
I'll note that the current release of pvl (0.3.0, i.e. before I went and overhauled it), also does not parse this file, so we didn't regress.
i'm pretty sure that's an error in the scripts of the PDS note, not of the ISS team, because it looks like the PDS team is re-applying the newer ISSCAL to previously delivered ISS data (often per my request ;) ), and the time format the ISS team usually used is "%Y-%j", it's even used in the other time stamps of this label, APART from the product creation time which I bet was a re-run on the PDS node.
and, because the calibrated files don't have an index file, i cannot show how prevalent it is, but manual browsing shows there's quite a few. I think a simple email to Rob French should suffice.