Add api registry and allow it to be added into client config in data source plugin #5895

BionIT · 2024-02-18T06:49:27Z

Description

This change allows consumers of data source plugin to register the API schema to be used in OpenSearch client and legacy client.

Issues Resolved

Fixes #5854

Screenshot

5854.mp4

Testing the changes

Here are the steps which was performed in the screenshot

disable data source plugin
add sample data from local cluster, and checked dashboard displays as expected
enable data source plugin
check the dashboard added from remote data source, and verified dashboard displays as expected
go to query bench which was changed locally to register the schema and use the new client, and verified that it can connect with remote data source and perform query as expected

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Lu Yu <nluyu@amazon.com>

codecov · 2024-02-18T07:03:12Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (55443f7) 66.98% compared to head (6d160b8) 66.98%.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5895   +/-   ##
=======================================
  Coverage   66.98%   66.98%           
=======================================
  Files        3304     3305    +1     
  Lines       63569    63574    +5     
  Branches    10153    10153           
=======================================
+ Hits        42580    42585    +5     
  Misses      18521    18521           
  Partials     2468     2468

Flag	Coverage Δ
Linux_1	`35.21% <ø> (ø)`
Linux_2	`55.11% <ø> (ø)`
Linux_3	`43.55% <100.00%> (+0.02%)`	⬆️
Linux_4	`35.20% <ø> (ø)`
Windows_1	`35.24% <ø> (ø)`
Windows_2	`55.07% <ø> (ø)`
Windows_3	`43.56% <100.00%> (+<0.01%)`	⬆️
Windows_4	`35.20% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

seraphjiang · 2024-02-18T20:57:22Z

@BionIT would you take a look https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/docs/multi-datasource/client_management_design.md

and update doc accordingly if necessary.

we could share the latest design doc to plugin team as reference.

Signed-off-by: Lu Yu <nluyu@amazon.com>

BionIT · 2024-02-19T02:13:06Z

@BionIT would you take a look https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/docs/multi-datasource/client_management_design.md

and update doc accordingly if necessary.

we could share the latest design doc to plugin team as reference.

Sure, just commit change to update the document

src/plugins/data_source/server/client/configure_client.ts

src/plugins/data_source/server/schema_registry/custom_api_schema_registry.ts

Signed-off-by: Lu Yu <nluyu@amazon.com>

src/plugins/data_source/server/plugin.ts

xinruiba · 2024-02-19T22:12:44Z

src/plugins/data_source/server/plugin.ts

@@ -155,7 +165,8 @@ export class DataSourcePlugin implements Plugin<DataSourcePluginSetup, DataSourc
    cryptography: CryptographyServiceSetup,
    logger: Logger,
    auditTrailPromise: Promise<AuditorFactory>,
-    authRegistryPromise: Promise<IAuthenticationMethodRegistery>
+    authRegistryPromise: Promise<IAuthenticationMethodRegistery>,
+    customApiSchemaRegistryPromise: Promise<CustomApiSchemaRegistry>


nit:
Overall LGTM approved, Thanks for this change~

If not complex, do we also want to create a ICustomApiSchemaRegistry type similar to IAuthenticationMethodRegistery to only expose register() also make the code format here consistent?

opensearch-trigger-bot · 2024-02-20T01:05:47Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch-Dashboards/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch-Dashboards/backport-2.x
# Create a new branch
git switch --create backport/backport-5895-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 eff7cb5536a552692c43fedd60fb8a9f87f1e801
# Push it to GitHub
git push --set-upstream origin backport/backport-5895-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch-Dashboards/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-5895-to-2.x.

…source plugin (opensearch-project#5895) * add api registry and allow it to be added into client config Signed-off-by: Lu Yu <nluyu@amazon.com> * add changelog Signed-off-by: Lu Yu <nluyu@amazon.com> * add documentation for multi data source plugin api registry Signed-off-by: Lu Yu <nluyu@amazon.com> * change to resolve promise before calling getQueryClient Signed-off-by: Lu Yu <nluyu@amazon.com> --------- Signed-off-by: Lu Yu <nluyu@amazon.com>

…source plugin (#5895) (#5906) * add api registry and allow it to be added into client config * add changelog * add documentation for multi data source plugin api registry * change to resolve promise before calling getQueryClient --------- Signed-off-by: Lu Yu <nluyu@amazon.com>

manasvinibs · 2024-03-13T20:18:01Z

Manual Backport #5906

add api registry and allow it to be added into client config

e365670

Signed-off-by: Lu Yu <nluyu@amazon.com>

BionIT requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, kristenTian, zhongnansu, manasvinibs, ZilongX, Flyingliuhub, BSFishy, curq, bandinib-amzn and SuZhou-Joe as code owners February 18, 2024 06:49

github-actions bot added the repeat-contributor label Feb 18, 2024

BionIT changed the title ~~Add api registry and allow it to be added into client config~~ Add api registry and allow it to be added into client config in data source plugin Feb 18, 2024

add changelog

8090abb

Signed-off-by: Lu Yu <nluyu@amazon.com>

add documentation for multi data source plugin api registry

f46f0de

Signed-off-by: Lu Yu <nluyu@amazon.com>

zengyan-amazon reviewed Feb 19, 2024

View reviewed changes

src/plugins/data_source/server/client/configure_client.ts Outdated Show resolved Hide resolved

src/plugins/data_source/server/schema_registry/custom_api_schema_registry.ts Show resolved Hide resolved

change to resolve promise before calling getQueryClient

6d160b8

Signed-off-by: Lu Yu <nluyu@amazon.com>

zengyan-amazon approved these changes Feb 19, 2024

View reviewed changes

seraphjiang approved these changes Feb 19, 2024

View reviewed changes

Flyingliuhub reviewed Feb 19, 2024

View reviewed changes

src/plugins/data_source/server/plugin.ts Show resolved Hide resolved

Flyingliuhub approved these changes Feb 19, 2024

View reviewed changes

xinruiba reviewed Feb 19, 2024

View reviewed changes

xinruiba approved these changes Feb 19, 2024

View reviewed changes

ZilongX approved these changes Feb 19, 2024

View reviewed changes

ZilongX merged commit eff7cb5 into opensearch-project:main Feb 19, 2024
65 of 67 checks passed

ZilongX added multiple datasource multiple datasource project v2.13.0 backport 2.x labels Feb 19, 2024

opensearch-trigger-bot bot added the failed backport label Feb 20, 2024

BionIT mentioned this pull request Feb 21, 2024

[Manual Backport 2.x] Add api registry and allow it to be added into client config in data source plugin #5906

Merged

7 tasks

bandinib-amzn mentioned this pull request Feb 26, 2024

[BUG] Test Connection is not working for multi data source #5924

Closed

BionIT mentioned this pull request Mar 5, 2024

[Meta][2.13] Security, bug fix and enhancemanet for dashboards anywhere #5752

Open

28 tasks

manasvinibs removed the failed backport label Mar 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add api registry and allow it to be added into client config in data source plugin #5895

Add api registry and allow it to be added into client config in data source plugin #5895

BionIT commented Feb 18, 2024 •

edited

Loading

codecov bot commented Feb 18, 2024 •

edited

Loading

seraphjiang commented Feb 18, 2024

BionIT commented Feb 19, 2024

xinruiba Feb 19, 2024 •

edited

Loading

opensearch-trigger-bot bot commented Feb 20, 2024

manasvinibs commented Mar 13, 2024

Add api registry and allow it to be added into client config in data source plugin #5895

Add api registry and allow it to be added into client config in data source plugin #5895

Conversation

BionIT commented Feb 18, 2024 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Check List

codecov bot commented Feb 18, 2024 • edited Loading

Codecov Report

seraphjiang commented Feb 18, 2024

BionIT commented Feb 19, 2024

xinruiba Feb 19, 2024 • edited Loading

Choose a reason for hiding this comment

opensearch-trigger-bot bot commented Feb 20, 2024

manasvinibs commented Mar 13, 2024

BionIT commented Feb 18, 2024 •

edited

Loading

codecov bot commented Feb 18, 2024 •

edited

Loading

xinruiba Feb 19, 2024 •

edited

Loading