Swirrl / table2qb

A generic pipeline for converting tabular data into rdf data cubes

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Allignment between pipelines

zeginis opened this issue · comments

Working with Table2qb tha main issue I faced is the allignment between the pipelines.

There are 3 pipelines and the URIs created should be aligned manually. The user should know exactly the output URIs created by the codelist-pipeline and components-pipeline and use them as input at the configuration of the cube-pipeline. e.g.

  • the property_template URI at cube-pipeline should match with the URIs created for the components by the components-pipeline
  • the value_template URI at the cube-pipeline match with the URIs created by the codelist-pipeline

This is also related to #84

Sorry, I don't understand what you're suggesting. Perhaps that we improve the documentation to make the link clear?

We can't align these in code (e.g. specifying a property_template along with the component, then referring to this with a var name) because:

  • You can use components in the cube-pipeline that you didn't create in the components-pipeline (e.g. a external dimension property like sdmx-dim:refPeriod)
  • There isn't a 1:1 relation between column:component, allowing for "overloading" of a given component (so you can create different value-templates for the same property-template) or the derivation of a property-template from multiple columns in the input

I think the best way to ensure alignment is with validation of the results - e.g. making sure that a graph combining outputs from the various pipelines (plus any external vocabularies) has resources that all join-up correctly.

I don't have a clear suggestion. Just wanted to report an issue we faced using Table2qb.

Ok then a validation and an improvement of the documentation will be helpfull.

The new usage doc from PR #95 should help to explain the alignment and (once implemented) #97 will help demonstrate/ validate referential integrity. Further #99 should help with some more examples.

Thanks