Separate reference docs from manual pages

[bentsherman]

This PR reorganizes the docs in a number of ways:

• reorganize manual pages into "Running pipelines" and "Developing pipelines" to be more aligned with the user journey
• separate reference content (config scopes, process directives, etc) from the main docs
• improve section on managing dependencies

The original exploration was around refactoring the docs into a user guide vs reference manual. After a few iterations, the main focus now is the first two items, separating the reference docs and reorganizing the sections to be more user friendly.

[netlify]

✅ Deploy Preview for nextflow-docs-staging ready!

Name	Link
🔨 Latest commit	22a4a5d
🔍 Latest deploy log	https://app.netlify.com/sites/nextflow-docs-staging/deploys/66f3ffb2b0a8770008d60fcf
😎 Deploy Preview	https://deploy-preview-4766--nextflow-docs-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[bentsherman]

After discussing with Paolo and the team, we are thinking more to organize the docs like the Docker Docs.

The key implication for this PR is that there will not be one general "user guide" and "reference". Instead, there will be a "Guides" section which contains many guides targeted to specific use cases, such as a quickstart for each cloud provider, HPC schedulers, etc.

The existing "reference" docs are more accurately described as a "manual" which also contains reference material. Instead of trying to factor out the user guide content, I will keep the docs as they are but just try to better separate the reference material from the "manual" pages.

I will refactor this PR accordingly and continue to restructure the existing docs, but also carve out a space for the team to start writing the guides. These efforts can be pursued independently, but since they both depend on a new deployment framework, no harm in keeping them together for now.

[bentsherman]

Some notes on the Nextflow training and nf-core docs after a brief review:

• nf-core docs seems to be mostly about nf-core conventions, really only the installation page has some duplication that could be trimmed

• the advanced and "applied" training guides can be adapted into guides under our proposed structure, pretty straightforward

• the basic training duplicates a lot of nextflow docs, it should be condensed significantly and become another guide (see notes below)

Because I have pulled out the reference material (config scopes, channel factories, process directives, etc), the manual pages for config, channels, process, etc have a lot more breathing room to give some illustrative examples. As a result, I think the basic training can be condensed significantly by just linking to the docs more instead of duplicating it. Instead it can focus on walking through the writing and testing of a specific pipeline example.

[bentsherman]

After discussing with Paolo, we would like to only keep the manual + reference docs in the Nextflow repo. Additional content like targeted guides and tutorials should be maintained somewhere else and can be combined with the reference docs at build time.

To that end I have trimmed this PR to focus on the internal improvements to the existing docs. These changes can be merged independently of other efforts like writing guides, deployment framework, website overhaul, etc.

[bentsherman]

@pditommaso this PR is up to date. Let's get it merged so that we don't have to keep fixing conflicts

[pditommaso]

@bentsherman have a look at those conflicts please

[christopher-hakkaart]

The structure feels good and I found it intuitive to find the content I was looking for.

I've left some comments/suggestions that tidy up some inconsistent language and formatting in close proximity on a single page.

A second sanity check would be worth while but otherwise I think it's good to go.

[pditommaso]

Finally managed to went through. It looks excellent. A few comments:

• Fusion docs looks updated. It should be aligned to the upstream
• Plugins would make more sense in "Running pipelines", with the exception of "Wring plugins" that should go into "Contributing"
• Is Workflow Diagram broken?

[bentsherman]

Fusion docs looks updated. It should be aligned to the upstream

done

Plugins would make more sense in "Running pipelines", with the exception of "Wring plugins" that should go into "Contributing"

I have reorganized these pages to better separate the user vs developer content. Both pages could be improved and expanded further, but I'd rather save that for a separate PR. For now I just reorganized the content that was already there.

Is Workflow Diagram broken?

it's working for me locally and in the netlify preview

[christopher-hakkaart]

Plugin sections changes look good to me. A few minor language suggestions.