OpenRefine (previously Google Refine) is a powerful tool for working with messy data: cleaning it; transforming it from one format into another; and extending it with web services and external data.
The openrefine-wrench commandline tool supports the basic openrefine operations:
- create openrefine project (csv and xml source data only)
- apply rules to openrefine project
- export modified openrefine project (csv export only)
- delete openrefine project
The main feature of openrefine-wrench is the orchestration of these operations over multiple processes to handle multiple input files in separte openrefine projects.
$ openrefine-wrench --help
Usage: openrefine-wrench [OPTIONS]
Handle multiple input files in separte openrefine projects.
Options:
--host TEXT openrefine host [required]
--port TEXT openrefine port (default to 3333)
[required]
--source-dir TEXT openrefine source data dir [required]
--export-dir TEXT openrefine export data dir [required]
--source-format [xml|csv] openrefine source data format [required]
--encoding TEXT openrefine source data encoding (default to
UTF-8) [required]
--record-path TEXT record path (only applicable in conjunction
with xml source format)
--columns-separator TEXT columns separator (only applicable in
conjunction with csv source format)
--mappings-file TEXT openrefine mappings file [required]
--max-workers INTEGER number of parallel processed openrefine
projects [required]
--log-level [DEBUG|INFO|WARN|ERROR|OFF]
log level (default INFO)
--custom-options TEXT custom options (overrides everything, only
in case you know what you're doing)
--logfile TEXT openrefine-wrench related logfile
--help Show this message and exit.
$ openrefine-wrench-create --help
Usage: openrefine-wrench-create [OPTIONS]
Create single openrefine project.
Options:
--host TEXT openrefine host [required]
--port TEXT openrefine port (default to 3333)
[required]
--source-file TEXT openrefine source file [required]
--source-format [xml|csv] openrefine source data format [required]
--project-name TEXT openrefine project name [required]
--encoding TEXT openrefine source data encoding (default to
UTF-8) [required]
--record-path TEXT record path (only applicable in conjunction
with xml source format)
--columns-separator TEXT columns separator (only applicable in
conjunction with csv source format)
--log-level [DEBUG|INFO|WARN|ERROR|OFF]
log level (default INFO)
--custom-options TEXT custom options (overrides everything, only
in case you know what you're doing)
--logfile TEXT openrefine-wrench-create related logfile
--help Show this message and exit.
$ openrefine-wrench-apply --help
Usage: openrefine-wrench-apply [OPTIONS]
Apply rules to single openrefine project.
Options:
--host TEXT openrefine host [required]
--port TEXT openrefine port (default to 3333)
[required]
--project-id TEXT openrefine project id [required]
--mappings-file TEXT openrefine mappings file [required]
--log-level [DEBUG|INFO|WARN|ERROR|OFF]
log level (default INFO)
--logfile TEXT openrefine-wrench-apply related logfile
--help Show this message and exit.
$ openrefine-wrench-export --help
Usage: openrefine-wrench-export [OPTIONS]
Export single modified openrefine project.
Options:
--host TEXT openrefine host [required]
--port TEXT openrefine port (default to 3333)
[required]
--export-file TEXT openrefine export file [required]
--project-id TEXT openrefine project id [required]
--log-level [DEBUG|INFO|WARN|ERROR|OFF]
log level (default INFO)
--logfile TEXT openrefine-wrench-export related logfile
--help Show this message and exit.
$ openrefine-wrench-delete --help
Usage: openrefine-wrench-delete [OPTIONS]
Delete single openrefine project.
Options:
--host TEXT openrefine host [required]
--port TEXT openrefine port (default to 3333)
[required]
--project-id TEXT openrefine project id [required]
--log-level [DEBUG|INFO|WARN|ERROR|OFF]
log level (default INFO)
--logfile TEXT openrefine-wrench-delete related logfile
--help Show this message and exit.
Install via pip, e.g.:
$ python3 -m pip install git+https://github.com/slub/openrefine-wrench --user
This repository is licensed under the MIT License.