No relative time ranges on Unidata IDD FMRC datasets
ethanrd opened this issue · comments
The Unidata IDDs FMRC datasets no longer (if they ever did) have time ranges relative to present time. Here's an example I grabbed from one of the datasetScan's in the satellite catalog:
<timeCoverage>
<end>present</end>
<duration>7 days</duration>
</timeCoverage>
I don't know if the FMRCs were ever configured like this. But they should be. (I think.)
As far as I know, we only have a single test FMRC on thredds-test, and it does have the timeCoverage element as you describe above:
http://thredds-test.unidata.ucar.edu/thredds/catalog/fmrc/ecmwf/global_2p5/catalog.xml
It's explicitly listed in the config catalog metadata and inherited by the FMRC, but it does show through in the client xml.
Should we update each of the FMRCs on thredds.ucar.edu to have a relative time range?
A bit of a maintenance headache as the length of data kept does change occasionally.
If we don't update these now because of the maintenance headache, we should plan (at some point down the line) to add a way to configure FMRCs to check the time extent and insert a relative time range.
So with the new Grib featureCollections, the time coverage is actually computed from the data in the collection. Here is what we have right now for the GFS 0.25 degree forecast collection:
top level catalog metadata:
<timeCoverage>
<end>present</end>
<duration>45 days</duration>
</timeCoverage>Yuck.
full collection
<timeCoverage>
<start>2017-09-16T00:00:00Z</start>
<end>2017-10-04T00:00:00Z</end>
</timeCoverage>best
<timeCoverage>
<start>2017-09-16T00:00:00Z</start>
<end>2017-10-04T00:00:00Z</end>
</timeCoverage><timeCoverage>
<start>2017-09-18T00:00:00Z</start>
<end>2017-10-04T00:00:00Z</end>
</timeCoverage>The top level catalog metadata is incorrect, and is coming from an explicit entry in our config catalog. I wonder if we commented out the explicit entry if the Grib featureCollection would include the right information (essentially the "full" collection time coverage).
That said, are you saying you'd rather see a relative time coverage? For this collection, something like:
<timeCoverage>
<end>present</end>
<duration>2 days</duration>
</timeCoverage>In this case, time coverage would not include forecast times, just the range of run times.
Yes, I am saying it would be nice to accurately describe the time range of all our IDD data sets as a rolling time-window rather than an absolute range. It has always been a maintenance nightmare (i.e., almost always wrong) and I suspect will continue to be until this and our scour scripts are using the same configuration. Though, I suppose in the mean time, we could automatically calculate a duration for each dataset on server startup (or once a week, or something).
Yes, just the range of run times, not forecast times.
This is not a big priority as I don't know of any dataset search service that understands non-absolute time ranges. However, we've been trying to communicate this use case to the dataset search community since the beginning of THREDDS without much success. It would be nice to have some datasets that actually use this, so we can point to it as an example.