Add custom script engines for donwloading/uplaod/listing

[tmspzz]

See #146.

It should also work for: #120, #139, #116

[jacobh0]

Any word on this PR? 👍 for getting it in. We need to add caching support, but have on our on-site file store that we would like to use instead of S3.

[tmspzz]

@grioja I work on this on and off, contributions are welcome.

In the mean time, setting up minio https://www.minio.io/ might get you around the problem.

[BalestraPatrick]

@blender @fabb What's missing to land this PR? I may try to help out a little bit with my very little knowledge of Haskell if something is needed.

[tmspzz]

@BalestraPatrick basically the path is laid out. What is missing is the implementation.

Whenever you read undefined, that is where the gap needs filling.

[tmspzz]

I have resolved the conflicts are rebased on #179

[BalestraPatrick]

Can you elaborate a little bit on the idea for your implementation?

I did actually get to fix the conflicts locally but didn't push them, oops. Happy to see you had time to do it!

[hlintBot]

	1 Warning
⚠️	Big PR

Generated by 🚫 Danger

[tmspzz]

@BalestraPatrick the idea is to invoke a script at mEnginePath and pass it a bunch of arguments:

First argument is always the command currently being executed: i.e. download, upload or list

In case of download, arguments are:

1. download
2. remote-path-to-the-file-as-constructed-by-rome
3. local-path-as-constructed-by-rome

In case of upload, arguments are:

1. upload
2. local-path-as-constructed-by-rome
3. remote-path-to-the-file-as-constructed-by-rome

In case of list, arguments are:

1. list
2. remote-path-to-the-file-as-constructed-by-rome

scripts should return exit code 0 on success and be passed stdin, stdout and stderr descriptors by rome itself. You can use Turtle to invoke the scripts.

Important: the script is going to be invoked concurrently so it should either have no state or support current execution.

[tmspzz]

This is to start with so that things don't get too complicated.

Alternatively the idea was to pass a description of the data structure that the commands rely upon to the script. However I don't have a concise solution for this. One idea is to use JSON to describe them but I think for a first implementation this is an overkill.

Eventually if all the contextual information that the commands have is required also in the script (and cannot be extracted from the paths) then I'll reconsider.