Add resource size info to transfer type OCM share

[redblom]

At WP4.4 it is discussed that it would be useful for a receiver of a transfer type OCM share to know the size of the transfer before accepting it. Although the size cannot be guaranteed, a sender could still add more files to a transfer folder which would also be transferred after acceptance in case of a transfer retry, the receiving end could limit the transfer to the expected size (data cap). Or, from the end user viewpoint, have an idea of what to expect.
For the receiver it is not possible to stat the resource to be transferred before accepting it because the webdav reference to it is only created after accepting the share. Thus, the size info must be send by the creator of the resource as a specific size parameter.

The initiator of the datatx type OCM share adds the current size to the opaque field of the request:

reva/cmd/reva/transfer-create.go

Line 112 in 44c9fa1

createShareReq := &ocm.CreateOCMShareRequest{

		createShareReq := &ocm.CreateOCMShareRequest{
			Opaque: &types.Opaque{
				Map: map[string]*types.OpaqueEntry{
					"permissions": {
						Decoder: "plain",
						Value:   []byte(strconv.Itoa(pint)),
					},
					"name": {
						Decoder: "plain",
						Value:   []byte(statRes.Info.Path),
					},
					"protocol": {
						Decoder: "plain",
						Value:   []byte("datatx"),
					},
					"size": {
						Decoder: "plain",
						Value:   []byte("60000000"),
					},
				},
			},

Next we add the size as an option of the protocol section of the OCM share request:

reva/pkg/ocm/share/manager/json/json.go

Line 257 in 44c9fa1

protocol := map[string]interface{}{

		protocol := map[string]interface{}{
			"name": "webdav",
			"options": map[string]string{
				"permissions": pm,
				"token":       ctxpkg.ContextMustGetToken(ctx),
				"size": size,
			},
		}

And the datatx driver will be called specifying the size in a new options parameter:

reva/pkg/datatx/datatx.go

Line 30 in 44c9fa1

StartTransfer(ctx context.Context, srcRemote string, srcPath string, srcToken string, destRemote string, destPath string, destToken string) (*datatx.TxInfo, error)

type Manager interface {
	// StartTransfer initiates a transfer job and returns a TxInfo object including a unique transfer id, and error if any.
	StartTransfer(
		ctx context.Context, 
		srcRemote string, 
		srcPath string, 
		srcToken string, 
		destRemote string, 
		destPath string, 
		destToken string, 
		options map[string]interface{}) (*datatx.TxInfo, error)
	...

In the driver implementation the decision to cap on size is based on the value of 2 new datatx config flags:

# if true and size option (set in bytes) is specified in the received datatx type share, the transfer will stop if that size has been reached
limit_transfer = true/false
# will stop the transfer if max_transfer_size (set in bytes) has been reached and limit_transfer is set to true
max_transfer_size = 60000000

[redblom]

@glpatcern @michielbdejong @gmgigi96 do we regard discovering the size of the transfer a task of reva or the efss ? (ps. I know its a required field, but are we aiming to keep reva as light as possible in that regard or spare the efss ?)

[glpatcern]

@redblom well someone would have to traverse the tree to be sent out and execute du on it: if it is the Reva gateway implementing the loop, the Reva storage providers (and the underlying EFSSs) have no extra requirement as it will just be a bunch of stat calls. Otherwise, each storage provider (or ultimately the EFSS? but there's no such functionality via WebDAV AFAIK) would have to implement such a discovery.

I'd implement it in the Reva gateway, factored out for everyone.

[redblom]

Implemented with #3891