Clarity European Data - T2.2 - CKAN Data Collection
ghilbrae opened this issue · comments
We've tested the information provided in CKAN for the European wide data that is either used by several DCs or by the European Data Package with special interest on the URLs provided for the Resources in the Datasets. The results can be seen in the attached file with the following structure:
DATASET: {dataset_name}
RESOURCE: {resource_name} (if there's a name)
URL {ERROR | EXISTS} -> {url} (if there's one)
We are aware that there are different people or institutions in charge of these data, so lease take a look at it and ensure that the URLs and data are correct for the ones that may be yours.
I've gone through the datasets and resources and have identified the ones that are wrong. By wrong I mean several possibilities:
- Missing URL
- Existing URL that does not work
- Existing URL that does not lead to the data
I'm attaching this new version here, please take a look at it. Most of the problematic resources are those related to climate indexes.
Regarding the climate indices from ZAMG: in CKAN we have listed all the climate indices, which we had calculated at the time. We have to go through again and add all climate indices that we have now. We will also use your txt files and add a URL where it is missing.
So far, the URL is pointing to the Zenodo-Clarity webpage, but there is no data yet.
We have agreed with Pascal to do so, in order to be able to provide a list and the future place where to find the data.
By the time we have uploaded the data, we'll insert the correct link.
Thanks, Claudia!
In the attached sheet I've made an analysis of the data available in CKAN and added some comments on its status.
Please take a look at it and address the comments. I have some doubts about some datasets because they seem to cover all Europe but the link to WMS contains a bbox that covers only Naples and I don't know if this is intended for some reason or if we should change either the bbox, the name, the group to which these datasets belong to (move them to DC1, maybe?). I think these layers are related to what @negroscuro did some time ago.
If there's data no longer in use, please delete the datasets.
Also, if you have changed anything in the last couple of days, those changes may not be reflected there yet, I've been working on verifying the data for a few days now. Please, let me know and update the comments anyway.
@ghilbrae I have fixed the problems with the 1971-2000 datasets in your table, as well as the datasets hot-days and snow-days.
I also noticed that I hadn't uploaded the new snow-days data to the clarity ftp. I have now put it under:
/clarityftp/europe/hazard_indices/precipitation/snow-days
Just a reminder that once @negroscuro has finished with the calculations for the final layers we have to update Zenodo with the correct layers and also the corresponding layers in CKAN (mainly changing bbox's of layers that are currently just showing Naples and should show Europe).
The following datasets in CKAN are missing the mandatory assignment to one of the 4 groups:
@negroscuro @ghilbrae Please update datasets.
Hi @p-a-s-c-a-l, sorry, but I wasn't aware of these groups. :-(
I have updated them for the "Hot days" and "ice days".
Please have a look at the Data Management Validation Report and correct all errors.
@p-a-s-c-a-l, @ghilbrae, I have updated some entries in CKAN and entered one missing index on Tuesday (now 3 wind indices with correct link to Zenodo: Fmax, Torro17, 98percentile of daily max wind speed). Just that you know.
Updated validation report: clarity.docx
Pan-European hazard datasets are fine, Hazard Local Effects Input Layers will be updated when all layers have been recalculated (again).
There are still some issues related to the Resccue Data Package and several resources related to DC4, e.g.