kwenzig / data-specification

Build and validate Tabular Data Packages for ddionrails studies

data-specification

Installation

Install library from GitHub:

$ pip install git+https://github.com/paneldata/data-specification

Or using pipenv:

$ pip install --upgrade pipenv
$ pipenv shell
$ pipenv install git+https://github.com/paneldata/master@datapackage-directory#egg=ddionrails_datapackage

Usage

Build a datapackage from a configuration file.

# Build a datapackage.json for soep-core
$ git clone https://github.com/paneldata/data-specification.git
$ cd data-specification/
$ ddionrails-datapackage build examples/soep-core/config.yml

Infer a datapackage from a directory containing metadata files.

# creates a file called datapackage.json in the current directory
$ ddionrails-datapackage infer metadata-directory

$ cd metadata-directory/
$ ddionrails-datapackage infer .

# providing a filename for the datapackage is possible
$ ddionrails-datapackage infer metadata-directory something-different.json

# try to use strict rules for all discovered metadata files
$ ddionrails-datapackage infer metadata-directory --strict

Validation

$ cd metadata-directory/

# validates all resources that are defined in datapackage.json
$ ddionrails-datapackage validate datapackage.json

# validates all resources that are defined in datapackage.json, including relationships
$ ddionrails-datapackage validate datapackage.json --check-relations

# validates "variables" resource that is defined in datapackage.json
$ ddionrails-datapackage validate datapackage.json variables

# validates "variables" resource that is defined in datapackage.json, including relationships
$ ddionrails-datapackage validate datapackage.json variables --check-relations