Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MD] Improved error handling for the search API when a null value is passed for the dataSourceId. #5882

Conversation

xinruiba
Copy link
Member

@xinruiba xinruiba commented Feb 16, 2024

Description

#5880

Good error handling is important to help our customer understand our production's expected behavior.

This is the PR to improve the error handling process by providing

  1. More accurate status code
  2. More descriptive error message

for the following scenario:

When MD is enabled, opensearch hosts not exist and search request send an empty datasource id. We should treat that request as a bad request with error code 400 instead of 500 along with a more descriptive error message.

Issues Resolved

Screenshot

N/A

Testing the changes

New test cases added

Check List

  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

…is empty, make error code to 400

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>
@xinruiba xinruiba changed the title [MD] Expose hosts in opensearch global config and when datasoruce id … [MD] When datasource id is empty error code should be 400 Feb 16, 2024
Signed-off-by: Xinrui Bai <xinruiba@amazon.com>
Copy link

codecov bot commented Feb 16, 2024

Codecov Report

Attention: Patch coverage is 83.33333% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 67.01%. Comparing base (6d5560e) to head (7f2616c).
Report is 1 commits behind head on main.

❗ Current head 7f2616c differs from pull request most recent head 657d954. Consider uploading reports for the commit 657d954 to get more accurate results

Files Patch % Lines
...ch/opensearch_search/opensearch_search_strategy.ts 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5882      +/-   ##
==========================================
- Coverage   67.01%   67.01%   -0.01%     
==========================================
  Files        3310     3307       -3     
  Lines       63647    63616      -31     
  Branches    10165    10164       -1     
==========================================
- Hits        42656    42633      -23     
+ Misses      18522    18514       -8     
  Partials     2469     2469              
Flag Coverage Δ
Linux_1 35.21% <0.00%> (-0.01%) ⬇️
Linux_2 55.09% <0.00%> (-0.01%) ⬇️
Linux_3 43.63% <83.33%> (-0.76%) ⬇️
Linux_4 35.19% <0.00%> (-0.01%) ⬇️
Windows_1 35.23% <0.00%> (-0.01%) ⬇️
Windows_2 55.05% <0.00%> (-0.01%) ⬇️
Windows_3 43.65% <83.33%> (-0.76%) ⬇️
Windows_4 35.19% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

CHANGELOG.md Outdated Show resolved Hide resolved
? context.core.opensearch.client.asCurrentUserWithLongNumeralsSupport
: context.core.opensearch.client.asCurrentUser;

return withDataSourceEnabled && request.dataSourceId && context.dataSource
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we assume when data source is enabled, then context.dataSource is never undefined or vice versa?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 @xinruiba Do you have scenario where withDataSourceEnabled is true but context.dataSource is undefined?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the comments~

Yes, we can assume when data source is enabled, then context.dataSource is never undefined or vice versa because:

  1. "dataSource" will always been registered as "registerRouteHandlerContext" in dataSource plugin setup which means once dataSource plugin's setup function get called, context.dataSource will never be undefined.
  2. If dataSource.enabled turns on, dataSource plugin's setup function will always get called.

In that case, withDataSourceEnabled is not needed in this decide client function. I will remove it.

Thanks

@xinruiba xinruiba changed the title [MD] When datasource id is empty error code should be 400 [MD] Improved error handling for the search API when a null value is passed for the dataSourceId. Feb 23, 2024
Signed-off-by: Xinrui Bai <xinruiba@amazon.com>
Comment on lines 200 to 208
await opensearchSearch.search(
(mockDataSourceEnabledContext as unknown) as RequestHandlerContext,
{
dataSourceId: 'Some dataSourceId',
}
);

expect(mockDataSourceApiCaller).toBeCalled();
expect(mockOpenSearchApiCaller).not.toBeCalled();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you correct this according to test description?

Copy link
Member Author

@xinruiba xinruiba Feb 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

callAsInternalUser: jest.fn(),
asScoped: jest.fn(),
config: {
hosts: [],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are passing host empty. Can you correct test case description? Also do we have test case when host is not defined, means it is commented? In that case it should take default value and call opensearch client

Copy link
Member Author

@xinruiba xinruiba Feb 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines 232 to 235
{
dataSourceId: '',
}
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have to pass empty? What if we don't pass at all? Can we cover both case pass empty or undefined value and doesn't pass at all

Copy link
Member Author

@xinruiba xinruiba Feb 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

await opensearchSearch.search(
(mockDataSourceEnabledContext as unknown) as RequestHandlerContext,
{
dataSourceId: 'Some dataSourceId',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought that you are testing send request without dataSourceid, but pass dataSourceId: 'Some dataSourceId'

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test cases updated, thanks for the comment.

@Flyingliuhub Flyingliuhub merged commit 7d77b9e into opensearch-project:main Feb 27, 2024
14 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Feb 27, 2024
…passed for the dataSourceId. (#5882)

* [MD] Using legacy client cofig instead of globle config

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] add test cases to test different hostconfig and datasourceId combinations of scenario

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* Update changelog file

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* Resolve comments

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] update test cases and resolve comments

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] Update unit test

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

---------

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>
(cherry picked from commit 7d77b9e)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
bandinib-amzn pushed a commit that referenced this pull request Feb 27, 2024
…passed for the dataSourceId. (#5882) (#5969)

* [MD] Using legacy client cofig instead of globle config



* [UT] add test cases to test different hostconfig and datasourceId combinations of scenario



* Update changelog file



* Resolve comments



* [UT] update test cases and resolve comments



* [UT] Update unit test



---------


(cherry picked from commit 7d77b9e)

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@xinruiba xinruiba deleted the xinrui_DataSourceIdRequired_WithNoHostConfig branch February 27, 2024 22:15
gaobinlong pushed a commit to gaobinlong/OpenSearch-Dashboards that referenced this pull request Feb 28, 2024
…passed for the dataSourceId. (opensearch-project#5882)

* [MD] Using legacy client cofig instead of globle config

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] add test cases to test different hostconfig and datasourceId combinations of scenario

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* Update changelog file

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* Resolve comments

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] update test cases and resolve comments

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] Update unit test

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

---------

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>
SuZhou-Joe pushed a commit to SuZhou-Joe/OpenSearch-Dashboards that referenced this pull request Mar 4, 2024
…passed for the dataSourceId. (opensearch-project#5882)

* [MD] Using legacy client cofig instead of globle config

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] add test cases to test different hostconfig and datasourceId combinations of scenario

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* Update changelog file

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* Resolve comments

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] update test cases and resolve comments

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

* [UT] Update unit test

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>

---------

Signed-off-by: Xinrui Bai <xinruiba@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants