umccrise: support S3 outputs

[pdiakumis]

Adding a bit of a walkthrough for parsing S3 results via presigned URLs, focusing on umccrise here.

• s3_files_list_filter_relevant: this is equivalent to gds_files_list_filter_relevant but it takes an S3 object directory as input instead of GDS. Surprisingly aws s3 ls --output json cannot generate JSON outputs (bad form from AWS here - Unable to get JSON output from aws s3 ls command aws/aws-cli#709) so had to go with aws --output json s3api list-objects-v2.

• s3_file_presignedurl: generates presigned URL for the given S3 object via aws s3 presign.

• s3_search: uses the portal API to search for the given file pattern under s3://umccr-primary-data-prod, e.g. s3_search("multiqc_data.json"), and returns the results in a tidy tibble.

  • There are alternatives to using the portal API; the boto3 Athena client is pretty cool but the outputs are a bit too messy for my liking, so I prefer RAthena which uses boto3 under the hood.

• the umccrise multi-sample reporter template now generates interactive plots for signature contributions, HRD across CHORD and HRDetect, and summarises the summary table from the cancer report across all samples (449 successful umccrise workflows with the required results on S3).