Use ext directive instead of custom extra directive

[bentsherman]

The float executor currently uses a custom process directive extra to apply float CLI options to specific processes:

process FOO {
  extra = '--option value'
}

However, this is using a hack by setting a directive with the equals sign. It's an unfortunate aspect of the current Nextflow DSL that this syntax happens to be allowed, but only the following syntax is intentionally supported:

process FOO {
  extra '--option value'
}

This is how users normally specify directives, but if they try this with extra then it will fail, because it is not defined as a directive in Nextflow.

I ignored it for a while because I see the value of this directive and couldn't provide an "official" way to implement it. But now I realize it can be done with the ext directive. This directive basically allows for custom directives, so you can use it to define process-specific CLI options, for example:

// nextflow.config
process {
  withName:'FOO' {
    ext.float = '--option value'
  }
}

// main.nf
process FOO {
}

This approach is much more aligned with what Nextflow users expect. Note also that the custom options are defined in the config file instead of the process definition, so a user can define them in a custom config file without having to modify the pipeline code.

Happy to review the code if you decide to implement it.

[bentsherman]

Maybe an even simpler solution is the clusterOptions directive, which is used by grid executors to apply custom CLI options. I guess I didn't think of it before because Float isn't really a cluster, even though it uses a CLI.

So you could use clusterOptions for now, and if you ever refactor to use an API (#24), you can use something like ext.float for that to distinguish CLI options from API options.

[jealous]

Thank you very much for the suggestion, Ben. However, at this time, I’m not sure we have the resources to implement it. Will let you know if we decide to move forward with this idea.

[bentsherman]

Of course it's up to you to decide when and how to implement it, since it's your plugin. But I also wanted to mention, we are working on a more strict syntax parser for Nextflow scripts, which will not support something like extra = ... in the process definition. You still have a long time to migrate before it becomes the default behavior in Nextflow, just giving you some advance notice

[jealous]

Thanks for the updates. So we will definitely do this sooner or later.

[jealous]

I took some time to look into this issue and it is not complex. So I fixed it and tested it. Looks good so far.

[jealous]

Change is available in nf-float 0.4.4