From a629e1371c37ea6a530f5dbad734f028af629e24 Mon Sep 17 00:00:00 2001 From: Fienne Date: Thu, 20 Oct 2022 19:55:27 +0100 Subject: [PATCH 1/4] edit hd5 to hdf5 and fix broken link --- src/topics/custom-types.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/topics/custom-types.md b/src/topics/custom-types.md index f0b59caf..e4c47a59 100644 --- a/src/topics/custom-types.md +++ b/src/topics/custom-types.md @@ -7,7 +7,7 @@ customisation/configuration of a tool/analysis without the need to fiddle with the CWL description directly. The example below is a CWL description of the [biom convert format][biom] tool for -converting a standard biom table file to hd5 format. +converting a standard biom table file to hdf5 format. ```{literalinclude} /_includes/cwl/custom-types/custom-types.cwl :language: cwl @@ -21,7 +21,7 @@ converting a standard biom table file to hd5 format. :name: custom-types.yml ``` -___Note:___ To follow the example below, you need to [download the example input file](https://raw.githubusercontent.com/common-workflow-language/user_guide/main/_includes/cwl/custom-types/rich_sparse_otu_table.biom), *rich_sparse_otu_table.biom* e.g. via `wget`: +___Note:___ To follow the example below, you need to [download the example input file](https://github.com/common-workflow-language/user_guide/blob/main/src/_includes/cwl/custom-types/rich_sparse_otu_table.biom), *rich_sparse_otu_table.biom* e.g. via `wget`: ```{code-block} console $ wget https://github.com/common-workflow-language/user_guide/raw/main/src/_includes/cwl/custom-types/rich_sparse_otu_table.biom @@ -48,7 +48,7 @@ the object is defined (`biom-convert-table.yaml`) and the name of the object within that file (`table_type`) that defines the custom type. In this case the `symbols` array from the imported `biom-convert-table.yaml` file define the allowable table options. For example, in `custom-types.yml`, we pass `OTU table` as an `input` that -tells the tool to create an OTU table in hd5 format. +tells the tool to create an OTU table in hdf5 format. The contents of the YAML file describing the custom type are given below: From 647240d6ab78941a1c92a72e6d817e646bdcbef3 Mon Sep 17 00:00:00 2001 From: Fienne Date: Fri, 21 Oct 2022 21:52:47 +0100 Subject: [PATCH 2/4] include shell command requirement in FAQ --- src/faq.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/src/faq.md b/src/faq.md index fb50e9ad..9aee203b 100644 --- a/src/faq.md +++ b/src/faq.md @@ -446,6 +446,23 @@ The reference runner and several other CWL implementations support running those Docker format containers using the Singularity engine. Directly specifying a Singularity format container is not part of the CWL standards. +## Shell Command Requirement +`ShellCommandRequirement` in CWL allows us to use `/bin/sh -c` in order to pass our command as a string to a shell intrepreter. `ShellQuote:false` leaves shell metacharacters unquoted, and thus they can still be interpreted and used by a shell. +From CommandLineTool.job(): + +``` +shellcmd, _ = self.get_requirement("ShellCommandRequirement") + if shellcmd is not None: + cmd = [] # type: List[Text] + for b in builder.bindings: + arg = builder.generate_arg(b) + if b.get("shellQuote", True): + arg = [shellescape.quote(a) for a in aslist(arg)] + cmd.extend(aslist(arg)) + j.command_line = ["/bin/sh", "-c", " ".join(cmd)] +``` +Without the `ShellCommandRequirement` the job will run with subprocess.Popen(shell=False) and thus no shell metacharacters can be used. + ## Debug JavaScript expressions You can use the --js-console option of cwltool, or you can try From d4da617043c26ab1f3e156d1744f3b15b1a65a0b Mon Sep 17 00:00:00 2001 From: Fienne Date: Mon, 24 Oct 2022 07:04:07 +0100 Subject: [PATCH 3/4] define shell command requirement --- src/faq.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/faq.md b/src/faq.md index 9aee203b..aed51242 100644 --- a/src/faq.md +++ b/src/faq.md @@ -447,6 +447,10 @@ those Docker format containers using the Singularity engine. Directly specifying a Singularity format container is not part of the CWL standards. ## Shell Command Requirement +`ShellCommandRequirement` in CWL is used to convert a list of arguments into a string containing a shell command line. +Each item in the argument list must be joined into a string separated by single spaces and quoted to prevent intepretation by the shell, unless `CommandLineBinding` for that argument contains `shellQuote: false`. + +If `shellQuote: false` is specified, the argument is joined into the command string without quoting, which allows the use of shell metacharacters such as `|` for pipes. `ShellCommandRequirement` in CWL allows us to use `/bin/sh -c` in order to pass our command as a string to a shell intrepreter. `ShellQuote:false` leaves shell metacharacters unquoted, and thus they can still be interpreted and used by a shell. From CommandLineTool.job(): From bfca842b90dad251a9e10019641f48f03c997318 Mon Sep 17 00:00:00 2001 From: Fienne Date: Mon, 24 Oct 2022 07:16:41 +0100 Subject: [PATCH 4/4] update shell command requirement --- src/faq.md | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/src/faq.md b/src/faq.md index aed51242..1efc3fd0 100644 --- a/src/faq.md +++ b/src/faq.md @@ -452,20 +452,9 @@ Each item in the argument list must be joined into a string separated by single If `shellQuote: false` is specified, the argument is joined into the command string without quoting, which allows the use of shell metacharacters such as `|` for pipes. `ShellCommandRequirement` in CWL allows us to use `/bin/sh -c` in order to pass our command as a string to a shell intrepreter. `ShellQuote:false` leaves shell metacharacters unquoted, and thus they can still be interpreted and used by a shell. -From CommandLineTool.job(): -``` -shellcmd, _ = self.get_requirement("ShellCommandRequirement") - if shellcmd is not None: - cmd = [] # type: List[Text] - for b in builder.bindings: - arg = builder.generate_arg(b) - if b.get("shellQuote", True): - arg = [shellescape.quote(a) for a in aslist(arg)] - cmd.extend(aslist(arg)) - j.command_line = ["/bin/sh", "-c", " ".join(cmd)] -``` -Without the `ShellCommandRequirement` the job will run with subprocess.Popen(shell=False) and thus no shell metacharacters can be used. +To use the `ShellCommandRequirement`, you need to pass in the class name `ShellCommandRequirement` as a string. + ## Debug JavaScript expressions