Proposed changes to fs-to-dc

[Paul Haesler]

See also #367

Again we are finding use cases where we would like to be able to pass multiple file paths to fs-to-dc rather than a base directory and an explicit glob pattern.

The concerns raised by @Kirill888 in #367 have some merit, but do not apply to the specific use cases we are looking at.

I propose the following backwards-compatible behaviour:

Add a new flag option --files:

• If not set (the default), fs-to-dc works as it does now - with an input directory and an optional --glob handled internally.
• If set, then explicit file paths are expected. The "input directory" parameter is interpreted as the first path, and additional paths can be supplied up to the maximum command line length.
• Setting both --files and --glob is an error.

I have reason to suspect that fs-to-dc is not heavily used. (When I updated it recently it had a serious bug that rendered it unusable.)

[Duncan Gray]

Overall adding the dataset maturity concept to "datacube dataset add" would be my favoured approach. I haven't ever used odc-tools on NCI and I would prefer keeping the code dependencies down.

[Damien Ayers]

That sounds like a good compromise, thank you Paul.