CWL 1.0 workflow updates

Overview

Learn about the workflow improvements in CWL v1.0 from CWL sbg:draft-2.

ScatterMethod

Scatter by multiple input ports and define how jobs are formed. Additinally, you can define how outputs are organized. Scatter methods include the following.

Scatter by single port

This method performs the same as in CWL sbg:draft-2.

Example:

scatter: A

dotproduct

This method specifies that each of the input arrays are aligned and one element is taken from each array to construct each job. An error is thrown if all input arrays are not the same length.

scatter: [A, B]
scatterMethod: "dotproduct"

nested_crossproduct

This method specifies the Cartesian product of the inputs, producing a job for every combination of the scattered inputs. The output must be a nested array for each level of scattering in the order in which the input arrays are listed in the scatter field.

scatter: [A, B]
scatterMethod: "nested_crossproduct"

flat_crossproduct

This method specifies the Cartesian product of the inputs, producing a job for every combination of the scattered inputs. The output arrays must be flattened to a single level but otherwise listed in the order in which the input arrays are listed in the scatter field.

scatter: [A, B]
scatterMethod: "flat_crossproduct"

Input port valueFrom field

This method allows you to connect any CWL type to a step input and reshape it to fit the tool input type.

Example1:

The tool expects a single file object and the step receives an array with one file object inside. Using valueFrom, the input can be reshaped from a list containing one file object to a single file object.

valueFrom: $(self[0])

Example2:

As shown here, the tool expects a string input (file name) and instead the step receives a file object. Using valueFrom, just the file basename is provided to the tool.

valueFrom: $(self.basename)

No more batch input

Defining a workflow input port for batching is not supported in CWL 1.0. However, tasks can be batched via the API or the visual interface on Rabix.

Link merge

This method merges multiple inbound links into a single array. If not specified, the default method is "merge_nested".