option to suppress 'content-length' signing?

[cmbrehm]

I had a situation where I needed to POST to a custom API using AWS Signature v4, I used your library but it only was able to get it to work when I suppressed the signing of the 'content-length' header.

I did this by forking and deleting the two lines in aws4.js that had to do with applying the content-length header. I just wanted to let you know as I'm uncertain why this is happening (maybe intermediary filtering?) or if others were running into requirements to restrict the signed headers. I'm happy to spend some time adding a feature if you had any thoughts about how we could integrate this functionality back into your library.

Thanks for your thoughts, appreciate your work on this!

[mhart]

@cmbrehm when you say a custom API, do you mean one that's not Amazon's?

[cmbrehm]

@mhart right not it's an AWS service and I'm not sure how it's hosted, just that I need to use AWS Signature v4 to authenticate to it. It's possibly API Gateway(?) but could be just an independent implementation of the spec.

[mhart]

If it's just some custom API, then I'm not really keen to add yet-another-option to aws4 – because this is intended to support AWS' APIs.

It's kinda weird that they wouldn't sign the Content-Length header – it's a required header when using POST, unless you're using Transfer-Encoding: chunked – is that the case here?

[cmbrehm]

No, it's not chunked transfer. Also baffled why it wouldn't support that signing.

I understand the hesitation with adding to the API. I suspect this is run through API Gateway so let me run some test cases against that using your latest code and I'll get back to you.

[cmbrehm]

Michael,

Confirmed API Gateway works fine with content-length. I strongly suspect my root issue is a buggy implementation of the authentication specification by the API provider.

Feel free to close it, I'll live with the fork for now and try to work with the provider.

[mhart]

It's no problem for you to close it – you resolved it, all good! 👍

[omyani]

Hi,

I also stumbled into this very issue when interacting with AWS Elasticsearch (ES). The problem is that I can't even put my finger on when AWS accepts or rejects the signed payload.

When rejected, I get messages like:

'PUT
/search-utest-.../_bulk

content-length:
content-type:application/json 

as if AWS expects this field to be empty

Thanks!

[dblock]

I'd like to potentially reopen this. Currently the code adds a content-length based on the body and signs it. OpenSearch Serverless preview requires not to include content-length in signed headers. We're looking at fixing this on the server, but I think it's a valid ask to either select headers that are used for signing or be able to remove content-length.

See opensearch-project/opensearch-js#356 (comment) for details.

[mhart]

I've added support for this in v1.12.0 with the ability to specify headers to ignore (or specifically include if you need) during signing. They need to be lowercase object keys, mapping to a truthy value.

Eg:

aws4.sign({
  host: '07tjusf2h91cunochc.us-east-1.aoss.amazonaws.com',
  method: 'PUT',
  path: '/my-index',
  body: '{"mappings":{}}',
  headers: {
    'Content-Type': 'application/json',
    'X-Amz-Content-Sha256': 'UNSIGNED-PAYLOAD'
  },
  extraHeadersToIgnore: {
    'content-length': true
  }
})

[dblock]

That didn't seem to do it or I am using it wrong.

This code still re-adds the content-length header doesn't it?

      if (request.body && !headers['Content-Length'] && !headers['content-length'])
        headers['Content-Length'] = Buffer.byteLength(request.body)

You can try https://github.com/dblock/opensearch-node-client-demo/tree/opensearch-serverless-aws-sdk2, if you need a serverless instance to access, send me an account/role arn to dblock[at]dblock[dot]org, I should be able to add you to mine.

[mhart]

It does add the content-length header to the request – but the issue was about the content-length header being included in the signing

[mhart]

The official docs add the content-length to the request after signing too. What errors are you seeing? https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-clients.html#serverless-javascript

[dblock]

I'm getting a 403 with any body being sent. The doc says that you're supposed to remove content length. I think something may have changed server-side since that doc was written (it's a preview), I'm still tracking it down. Either way my goal is to get it to work the same way as Managed OpenSearch, I'll get back to this early this week.

So looks like the service just ignores any body/doesn't want it signed rn. Sending a signature of only headers works. dblock/opensearch-node-client-demo@9290669