common-workflow-language / common-workflow-language

Repository for the CWL standards. Use https://cwl.discourse.group/ for support 😊

Home Page:https://www.commonwl.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add standard attribute to pass paths of a list of files

opened this issue · comments

A common pattern that emerges (for example see https://cwl.discourse.group/t/how-to-iterate-over-input-file-array-from-arguments-valuefrom/79/2) is to loop over the paths in a list of files in a bash script. It would be nice to offer this as a property of a File list input, e.g.

inputs.files.paths

will be a string with paths in quotes, separated by spaces.

As discussed on a previous call, each addition to the standard imposes a burden on those implementing CWL executors. There is a tension between this cost and the benefit of adding features that support CWL authors. There are, as yet, no metrics or guidelines to help decide on this difference. Two metrics that come to mind are:

  1. How hard is this to implement? Here implementation in the cwltool reference runner could give some sense of the difficulty.
  2. What alternatives are there to having this feature? What are their downsides? And can CWL authoring environments implement an alternative solution (and at what cost)?

So while I personally think this is a useful suggestion, I'd like to offer the above metrics as a start towards coming up with some guidelines for changes for the standard.

We can also implement this as a standalone tool that expands the globs. That would work immediately with any CWL v1.x system.