Improvements for Collection Operations

[jmchilton]

• Tool form gives no indication of batch or mapping an input vs. not-batching an input if the input type is a data_collection. No Visual Indication of Mapping over a Collection #2467
• These tools readily produce list:list and such that the history panel does not render. Limited Collection Types Render in History #2495
• Right now the collection outputs place duplicated HDAs (with non-duplicated Datasets) in the History (like other tools) - this sucks and would no longer be needed if Simplified User-Facing Dataset Collection Model #1810 is complete - work on Simplified User-Facing Dataset Collection Model #1810 should revise this behavior as part of a broader fix for the collection abstraction's place in the history.
• Filter failed should be able to consume a list:paired and properly filter it.
• A detailed review and revision of the naming used by the tools (especially collection parts) should be performed - some of it doesn't make a ton of sense and while this may be hard to correct at the framework-level these tools provide a direct Python function to produce outputs and could be done better.
• Filter failing datasets isn't what one wants to do in case of a cluster failure, Rerunning a failed dataset collection element should substitute the failed element #2235 should be complete and the tool should be updated with a discussion of when to use what.
• The other collection operations should be implemented - including those in Collection Operations #1313 (grouping and filtering collections) as well as other definitively useful operations such as exploding collections, merging collections, and re-labelling collections with "expressions".
• Extend the filter failed operation with option to filter empty datasets. Various datatypes may have empty data files that aren't not in fact empty files (e.g. they may have structural elements or metadata but still be empty) and so a general purpose filtering tool that can consume metadata is also important (see above).

[nknox]

I agree with the naming issue in collections. It would be great to download individual files from a collection which matches the individual dataset name within the collection.

[jennaj]

A user request for the improvement idea "Extend the filter failed operation with option to filter empty datasets." above was made at Biostars. I wasn't sure if that was tracked in a distinct ticket yet (this ticket is older). Please help to link that in if one exists. Thanks!

What are others doing for a workaround? Or did I miss a new alternative method? It would be good to post it here if exists.

Post: https://biostar.usegalaxy.org/p/26364

[Mataivic]

@jennaj I'm the user :). I've looked a bit at the code today, hoping to be able to participate to this improvement. I didn't find a lot of stuff, but I let it in a comment in an issue I had made on GitHub (#5090, @jmchilton had answered) as the same time I asked on Biostars.