Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds a new export from the package for aws-sdk v3 #758

Merged
merged 14 commits into from
Apr 24, 2024

Conversation

tusharf5
Copy link
Contributor

@tusharf5 tusharf5 commented Apr 17, 2024

Description

Based on the conversation in this issue #757 , this PR adds a new way to import the AwsSigv4Signer as shown below.

const { AwsSigv4Signer } = require('@opensearch-project/opensearch/aws-v3');

By using a new import path, the sdk remains backwards compatible while also adding a way for clients to avoid getting the entire aws-sdk included in the final bundle which results in an additional 22MB.

I tested the code and it does reduce the bundle size with the new import path

image

The bundle size is still not good though because '@aws-sdk/credential-provider-node' dynamically imports all types of credential providers ini, ec2, environment, node, process, http etc which is about 1MB in size.

Would it make sense to have the client provide the provider based on the environment instead of relying on the default provider chain? This would be a breaking change but it would delegate the credential responsibility on the client eliminating the dependency on aws-sdk & @aws-sdk/credential-provider-node?

https://github.com/opensearch-project/opensearch-js/blob/main/lib/aws/AwsSigv4Signer.js#L74

We will have to throw an error on this (making it required) line so the client always provides a getCredential function and we can remove the dependency on a default provider (both v2 and v3) from our end.

The doc examples already use the getCredential function so I am guessing most of the clients would already be providing this function and not relying on the default getCredential.

https://opensearch.org/docs/latest/clients/javascript/index/#authenticating-with-amazon-opensearch-service-aws-signature-version-4

Issues Resolved

Closes #757

Check List

  • New functionality includes testing.
    • All tests pass
  • Linter check was successfull - yarn run lint doesn't show any errors
  • Commits are signed per the DCO using --signoff
  • Changelog was updated.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tushar Sharma <ts17995@gmail.com>
@tusharf5 tusharf5 marked this pull request as ready for review April 17, 2024 20:46
Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good stuff. Document new functionality/changes in README, USERS_GUIDE, update samples. Add tests.

CHANGELOG.md Outdated Show resolved Hide resolved
lib/aws/AwsSigv4Signer-sdk-v3.js Outdated Show resolved Hide resolved
@dblock
Copy link
Member

dblock commented Apr 17, 2024

Would it make sense to have the client provide the provider based on the environment instead of relying on the default provider chain? This would be a breaking change but it would delegate the credential responsibility on the client eliminating the dependency on aws-sdk & @aws-sdk/credential-provider-node?

It's a good idea, but I would explore this in a future/separate PR.

Signed-off-by: Tushar Sharma <ts17995@gmail.com>
@tusharf5
Copy link
Contributor Author

@dblock updated the PR with the requested changes

Copy link
Member

@dblock dblock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏 This looks great to me!

Let's get another pair of eyes, maybe @nhtruong and merge?

USER_GUIDE.md Show resolved Hide resolved
lib/aws/AwsSigv4Signer-sdk-v3.js Outdated Show resolved Hide resolved
@dblock
Copy link
Member

dblock commented Apr 18, 2024

@tusharf5 fix CI pls, there's some license headers missing, and a complaint about a new license, 0BSD. It's just a BSD variant so I think we can add it to the list, but I'll go double check and we already allow it in OpenSearch-Dasbhoards.

Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
@dblock
Copy link
Member

dblock commented Apr 18, 2024

The license workflow needs to have 0BSD added to it, https://github.com/opensearch-project/opensearch-js/actions/runs/8739773216/job/23986128639?pr=758.

Signed-off-by: Tushar Sharma <ts17995@gmail.com>
@tusharf5
Copy link
Contributor Author

@dblock fixed

@tusharf5
Copy link
Contributor Author

checking the test error

Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
@tusharf5
Copy link
Contributor Author

tusharf5 commented Apr 18, 2024

1 existing test was failing since aws-sdk and v3 ask were not installed so the error thrown was different and now they are part of dev dependency so the error has changed. I've updated the test. hopefully no more test failures. @dblock

Edit - For some reason I can't run tests locally, I get "ERR_REQUIRE_ESM" errors otherwise I would have caught & fixed these errors locally.

Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
Signed-off-by: Tushar Sharma <ts17995@gmail.com>
@tusharf5
Copy link
Contributor Author

Had to remove the peerDependency change since npm automatically installs peerDepenencies (yarn does not) and that changes the import result which fails the test. It should be working now.

@dblock dblock requested a review from nhtruong April 22, 2024 12:49
server.stop();
});
});
});
Copy link
Collaborator

@nhtruong nhtruong Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these tests only testing if they throw the correct errors, and not for the code paths where no errors are thrown? I think that's why Codecov is not happy.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would require mocking the import as the dependencies are not installed. While tap supports mocking imports, the project does not use the latest tap version which has breaking changes.

Doesn't seem like there is any other way to test dependency missing/present scenarios without mocking.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should solve this with integration tests IMO, which is a separate affair.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking of setting up different package.json for each test case like you have in test/bundlers/esbuild-test/package.json

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good idea. Do we want to hold this PR until then? Open an issue and do it later?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do it in a different PR so we can get this merged. I'll make an issue.

@dblock dblock merged commit e93977a into opensearch-project:main Apr 24, 2024
63 of 65 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dynamic import bundles entire aws-sdk
3 participants