Add note on scaling

[pinin4fjords]

PR checklist

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the pipeline conventions in the contribution docs
• If necessary, also make a PR on the nf-core/differentialabundance branch on the nf-core/test-datasets repository.
• Make sure your code lints (nf-core lint).
• Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
• Usage Documentation in docs/usage.md is updated.
• Output Documentation in docs/output.md is updated.
• CHANGELOG.md is updated.
• README.md is updated (including new tool citations and authors/contributors).

[github-actions]

nf-core lint overall result: Passed ✅

Posted for pipeline commit b38d216

+| ✅ 275 tests passed       |+

✅ Tests passed:

• files_exist - File found: .gitattributes
• files_exist - File found: .gitignore
• files_exist - File found: .nf-core.yml
• files_exist - File found: .editorconfig
• files_exist - File found: .prettierignore
• files_exist - File found: .prettierrc.yml
• files_exist - File found: CHANGELOG.md
• files_exist - File found: CITATIONS.md
• files_exist - File found: CODE_OF_CONDUCT.md
• files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
• files_exist - File found: nextflow_schema.json
• files_exist - File found: nextflow.config
• files_exist - File found: README.md
• files_exist - File found: .github/.dockstore.yml
• files_exist - File found: .github/CONTRIBUTING.md
• files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
• files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
• files_exist - File found: .github/workflows/branch.yml
• files_exist - File found: .github/workflows/ci.yml
• files_exist - File found: .github/workflows/linting_comment.yml
• files_exist - File found: .github/workflows/linting.yml
• files_exist - File found: assets/email_template.html
• files_exist - File found: assets/email_template.txt
• files_exist - File found: assets/sendmail_template.txt
• files_exist - File found: assets/nf-core-differentialabundance_logo_light.png
• files_exist - File found: conf/modules.config
• files_exist - File found: conf/test.config
• files_exist - File found: conf/test_full.config
• files_exist - File found: docs/images/nf-core-differentialabundance_logo_light.png
• files_exist - File found: docs/images/nf-core-differentialabundance_logo_dark.png
• files_exist - File found: docs/output.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/usage.md
• files_exist - File found: lib/NfcoreTemplate.groovy
• files_exist - File found: lib/Utils.groovy
• files_exist - File found: lib/WorkflowMain.groovy
• files_exist - File found: main.nf
• files_exist - File found: assets/multiqc_config.yml
• files_exist - File found: conf/base.config
• files_exist - File found: conf/igenomes.config
• files_exist - File found: .github/workflows/awstest.yml
• files_exist - File found: .github/workflows/awsfulltest.yml
• files_exist - File found: lib/WorkflowDifferentialabundance.groovy
• files_exist - File found: modules.json
• files_exist - File found: pyproject.toml
• files_exist - File not found check: Singularity
• files_exist - File not found check: parameters.settings.json
• files_exist - File not found check: pipeline_template.yml
• files_exist - File not found check: .nf-core.yaml
• files_exist - File not found check: bin/markdown_to_html.r
• files_exist - File not found check: conf/aws.config
• files_exist - File not found check: .github/workflows/push_dockerhub.yml
• files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
• files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
• files_exist - File not found check: docs/images/nf-core-differentialabundance_logo.png
• files_exist - File not found check: .markdownlint.yml
• files_exist - File not found check: .yamllint.yml
• files_exist - File not found check: lib/Checks.groovy
• files_exist - File not found check: lib/Completion.groovy
• files_exist - File not found check: lib/Workflow.groovy
• files_exist - File not found check: lib/nfcore_external_java_deps.jar
• files_exist - File not found check: .travis.yml
• nextflow_config - Config variable found: manifest.name
• nextflow_config - Config variable found: manifest.nextflowVersion
• nextflow_config - Config variable found: manifest.description
• nextflow_config - Config variable found: manifest.version
• nextflow_config - Config variable found: manifest.homePage
• nextflow_config - Config variable found: timeline.enabled
• nextflow_config - Config variable found: trace.enabled
• nextflow_config - Config variable found: report.enabled
• nextflow_config - Config variable found: dag.enabled
• nextflow_config - Config variable found: process.cpus
• nextflow_config - Config variable found: process.memory
• nextflow_config - Config variable found: process.time
• nextflow_config - Config variable found: params.outdir
• nextflow_config - Config variable found: params.input
• nextflow_config - Config variable found: params.validationShowHiddenParams
• nextflow_config - Config variable found: params.validationSchemaIgnoreParams
• nextflow_config - Config variable found: manifest.mainScript
• nextflow_config - Config variable found: timeline.file
• nextflow_config - Config variable found: trace.file
• nextflow_config - Config variable found: report.file
• nextflow_config - Config variable found: dag.file
• nextflow_config - Config variable (correctly) not found: params.nf_required_version
• nextflow_config - Config variable (correctly) not found: params.container
• nextflow_config - Config variable (correctly) not found: params.singleEnd
• nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
• nextflow_config - Config variable (correctly) not found: params.name
• nextflow_config - Config variable (correctly) not found: params.enable_conda
• nextflow_config - Config timeline.enabled had correct value: true
• nextflow_config - Config report.enabled had correct value: true
• nextflow_config - Config trace.enabled had correct value: true
• nextflow_config - Config dag.enabled had correct value: true
• nextflow_config - Config manifest.name began with nf-core/
• nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
• nextflow_config - Config dag.file ended with .html
• nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
• nextflow_config - Config manifest.version ends in dev: 1.5.0dev
• nextflow_config - Config params.custom_config_version is set to master
• nextflow_config - Config params.custom_config_base is set to https://github.com/nf-core/configs/master
• nextflow_config - Lines for loading custom profiles found
• nextflow_config - nextflow.config contains configuration profile test
• nextflow_config - Config default value correct: params.study_name= study
• nextflow_config - Config default value correct: params.study_type= rnaseq
• nextflow_config - Config default value correct: params.study_abundance_type= counts
• nextflow_config - Config default value correct: params.affy_cel_files_archive= null
• nextflow_config - Config default value correct: params.querygse= null
• nextflow_config - Config default value correct: params.observations_id_col= sample
• nextflow_config - Config default value correct: params.observations_type= sample
• nextflow_config - Config default value correct: params.observations_name_col= sample
• nextflow_config - Config default value correct: params.features_id_col= gene_id
• nextflow_config - Config default value correct: params.features_name_col= gene_name
• nextflow_config - Config default value correct: params.features_type= gene
• nextflow_config - Config default value correct: params.features_metadata_cols= gene_id,gene_name,gene_biotype
• nextflow_config - Config default value correct: params.features_gtf_feature_type= transcript
• nextflow_config - Config default value correct: params.features_gtf_table_first_field= gene_id
• nextflow_config - Config default value correct: params.affy_file_name_col= file
• nextflow_config - Config default value correct: params.affy_background= true
• nextflow_config - Config default value correct: params.affy_bgversion= 2
• nextflow_config - Config default value correct: params.affy_cdfname= null
• nextflow_config - Config default value correct: params.affy_build_annotation= true
• nextflow_config - Config default value correct: params.proteus_measurecol_prefix= LFQ intensity
• nextflow_config - Config default value correct: params.proteus_norm_function= normalizeMedian
• nextflow_config - Config default value correct: params.proteus_plotsd_method= violin
• nextflow_config - Config default value correct: params.proteus_plotmv_loess= true
• nextflow_config - Config default value correct: params.proteus_palette_name= Set1
• nextflow_config - Config default value correct: params.filtering_min_abundance= 1.0
• nextflow_config - Config default value correct: params.filtering_min_samples= 1.0
• nextflow_config - Config default value correct: params.exploratory_clustering_method= ward.D2
• nextflow_config - Config default value correct: params.exploratory_cor_method= spearman
• nextflow_config - Config default value correct: params.exploratory_n_features= 500
• nextflow_config - Config default value correct: params.exploratory_whisker_distance= 1.5
• nextflow_config - Config default value correct: params.exploratory_mad_threshold= -5
• nextflow_config - Config default value correct: params.exploratory_main_variable= auto_pca
• nextflow_config - Config default value correct: params.exploratory_assay_names= raw,normalised,variance_stabilised
• nextflow_config - Config default value correct: params.exploratory_final_assay= variance_stabilised
• nextflow_config - Config default value correct: params.exploratory_palette_name= Set1
• nextflow_config - Config default value correct: params.differential_file_suffix= .deseq2.results.tsv
• nextflow_config - Config default value correct: params.differential_feature_id_column= gene_id
• nextflow_config - Config default value correct: params.differential_fc_column= log2FoldChange
• nextflow_config - Config default value correct: params.differential_pval_column= pvalue
• nextflow_config - Config default value correct: params.differential_qval_column= padj
• nextflow_config - Config default value correct: params.differential_min_fold_change= 2.0
• nextflow_config - Config default value correct: params.differential_max_pval= 1.0
• nextflow_config - Config default value correct: params.differential_max_qval= 0.05
• nextflow_config - Config default value correct: params.differential_feature_name_column= gene_name
• nextflow_config - Config default value correct: params.differential_foldchanges_logged= true
• nextflow_config - Config default value correct: params.differential_palette_name= Set1
• nextflow_config - Config default value correct: params.deseq2_test= Wald
• nextflow_config - Config default value correct: params.deseq2_fit_type= parametric
• nextflow_config - Config default value correct: params.deseq2_sf_type= ratio
• nextflow_config - Config default value correct: params.deseq2_min_replicates_for_replace= 7
• nextflow_config - Config default value correct: params.deseq2_independent_filtering= true
• nextflow_config - Config default value correct: params.deseq2_lfc_threshold= 0
• nextflow_config - Config default value correct: params.deseq2_alt_hypothesis= greaterAbs
• nextflow_config - Config default value correct: params.deseq2_p_adjust_method= BH
• nextflow_config - Config default value correct: params.deseq2_alpha= 0.1
• nextflow_config - Config default value correct: params.deseq2_minmu= 0.5
• nextflow_config - Config default value correct: params.deseq2_vs_method= vst
• nextflow_config - Config default value correct: params.deseq2_shrink_lfc= true
• nextflow_config - Config default value correct: params.deseq2_cores= 1
• nextflow_config - Config default value correct: params.deseq2_vs_blind= true
• nextflow_config - Config default value correct: params.deseq2_vst_nsub= 1000
• nextflow_config - Config default value correct: params.limma_spacing= null
• nextflow_config - Config default value correct: params.limma_block= null
• nextflow_config - Config default value correct: params.limma_correlation= null
• nextflow_config - Config default value correct: params.limma_method= ls
• nextflow_config - Config default value correct: params.limma_proportion= 0.01
• nextflow_config - Config default value correct: params.limma_stdev_coef_lim= 0.1,4
• nextflow_config - Config default value correct: params.limma_winsor_tail_p= 0.05,0.1
• nextflow_config - Config default value correct: params.limma_lfc= 0
• nextflow_config - Config default value correct: params.limma_adjust_method= BH
• nextflow_config - Config default value correct: params.limma_p_value= 1.0
• nextflow_config - Config default value correct: params.gsea_permute= phenotype
• nextflow_config - Config default value correct: params.gsea_nperm= 1000
• nextflow_config - Config default value correct: params.gsea_scoring_scheme= weighted
• nextflow_config - Config default value correct: params.gsea_metric= Signal2Noise
• nextflow_config - Config default value correct: params.gsea_sort= real
• nextflow_config - Config default value correct: params.gsea_order= descending
• nextflow_config - Config default value correct: params.gsea_set_max= 500
• nextflow_config - Config default value correct: params.gsea_set_min= 15
• nextflow_config - Config default value correct: params.gsea_norm= meandiv
• nextflow_config - Config default value correct: params.gsea_rnd_type= no_balance
• nextflow_config - Config default value correct: params.gsea_make_sets= true
• nextflow_config - Config default value correct: params.gsea_num= 100
• nextflow_config - Config default value correct: params.gsea_plot_top_x= 20
• nextflow_config - Config default value correct: params.gsea_rnd_seed= timestamp
• nextflow_config - Config default value correct: params.gprofiler2_significant= true
• nextflow_config - Config default value correct: params.gprofiler2_measure_underrepresentation= false
• nextflow_config - Config default value correct: params.gprofiler2_evcodes= false
• nextflow_config - Config default value correct: params.gprofiler2_max_qval= 0.05
• nextflow_config - Config default value correct: params.gprofiler2_domain_scope= annotated
• nextflow_config - Config default value correct: params.gprofiler2_min_diff= 1
• nextflow_config - Config default value correct: params.gprofiler2_palette_name= Blues
• nextflow_config - Config default value correct: params.shinyngs_build_app= true
• nextflow_config - Config default value correct: params.shinyngs_shinyapps_account= null
• nextflow_config - Config default value correct: params.shinyngs_shinyapps_app_name= null
• nextflow_config - Config default value correct: params.shinyngs_guess_unlog_matrices= true
• nextflow_config - Config default value correct: params.gene_sets_files= null
• nextflow_config - Config default value correct: params.logo_file= docs/images/nf-core-differentialabundance_logo_light.png
• nextflow_config - Config default value correct: params.css_file= assets/nf-core_style.css
• nextflow_config - Config default value correct: params.citations_file= CITATIONS.md
• nextflow_config - Config default value correct: params.report_title= null
• nextflow_config - Config default value correct: params.report_author= null
• nextflow_config - Config default value correct: params.report_description= null
• nextflow_config - Config default value correct: params.report_scree= true
• nextflow_config - Config default value correct: params.report_round_digits= 4
• nextflow_config - Config default value correct: params.custom_config_version= master
• nextflow_config - Config default value correct: params.custom_config_base= https://github.com/nf-core/configs/master
• nextflow_config - Config default value correct: params.max_cpus= 16
• nextflow_config - Config default value correct: params.max_memory= 128.GB
• nextflow_config - Config default value correct: params.max_time= 240.h
• nextflow_config - Config default value correct: params.publish_dir_mode= copy
• nextflow_config - Config default value correct: params.validate_params= true
• files_unchanged - .gitattributes matches the template
• files_unchanged - .prettierrc.yml matches the template
• files_unchanged - CODE_OF_CONDUCT.md matches the template
• files_unchanged - LICENSE matches the template
• files_unchanged - .github/.dockstore.yml matches the template
• files_unchanged - .github/CONTRIBUTING.md matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
• files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
• files_unchanged - .github/workflows/branch.yml matches the template
• files_unchanged - .github/workflows/linting_comment.yml matches the template
• files_unchanged - .github/workflows/linting.yml matches the template
• files_unchanged - assets/email_template.html matches the template
• files_unchanged - assets/email_template.txt matches the template
• files_unchanged - assets/sendmail_template.txt matches the template
• files_unchanged - assets/nf-core-differentialabundance_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-differentialabundance_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-differentialabundance_logo_dark.png matches the template
• files_unchanged - docs/README.md matches the template
• files_unchanged - lib/NfcoreTemplate.groovy matches the template
• files_unchanged - .gitignore matches the template
• files_unchanged - .prettierignore matches the template
• files_unchanged - pyproject.toml matches the template
• actions_ci - '.github/workflows/ci.yml' is triggered on expected events
• actions_ci - '.github/workflows/ci.yml' checks minimum NF version
• actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
• readme - README Nextflow minimum version badge matched config. Badge: 23.10.0, Config: 23.10.0
• readme - README Zenodo placeholder was replaced with DOI.
• pipeline_todos - No TODO strings found
• pipeline_name_conventions - Name adheres to nf-core convention
• template_strings - Did not find any Jinja template strings (150 files)
• schema_lint - Schema lint passed
• schema_lint - Schema title + description lint passed
• schema_lint - Input mimetype lint passed: 'text/csv'
• schema_params - Schema matched params returned from nextflow config
• system_exit - No System.exit calls found
• actions_schema_validation - Workflow validation passed: branch.yml
• actions_schema_validation - Workflow validation passed: awstest.yml
• actions_schema_validation - Workflow validation passed: linting.yml
• actions_schema_validation - Workflow validation passed: ci.yml
• actions_schema_validation - Workflow validation passed: clean-up.yml
• actions_schema_validation - Workflow validation passed: fix-linting.yml
• actions_schema_validation - Workflow validation passed: awsfulltest.yml
• actions_schema_validation - Workflow validation passed: linting_comment.yml
• actions_schema_validation - Workflow validation passed: download_pipeline.yml
• actions_schema_validation - Workflow validation passed: release-announcements.yml
• merge_markers - No merge markers found in pipeline files
• modules_json - Only installed modules found in modules.json
• multiqc_config - 'assets/multiqc_config.yml' contains report_section_order
• multiqc_config - 'assets/multiqc_config.yml' contains export_plots
• multiqc_config - 'assets/multiqc_config.yml' contains report_comment
• multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
• multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
• multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
• modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

• nf-core/tools version 2.12.1
• Run at 2024-02-13 10:09:23

[WackerO]

LGTM! How does shiny scale with such large datasets? If necessary, maybe add a suggestion to turn that off as well?