Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix index mapping #384

Merged
merged 1 commit into from
Aug 4, 2022
Merged

Conversation

ylwu-amzn
Copy link
Collaborator

Signed-off-by: Yaliang Wu ylwu@amazon.com

Description

ML model content stored as text by mistake in index. This PR fixed index mapping and store model content as binary.

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@ylwu-amzn ylwu-amzn requested a review from a team August 2, 2022 16:45
@codecov-commenter
Copy link

codecov-commenter commented Aug 2, 2022

Codecov Report

Merging #384 (d11c539) into main (3a86a78) will decrease coverage by 0.66%.
The diff coverage is 62.06%.

@@             Coverage Diff              @@
##               main     #384      +/-   ##
============================================
- Coverage     92.09%   91.43%   -0.67%     
- Complexity      519      528       +9     
============================================
  Files            58       59       +1     
  Lines          1481     1529      +48     
  Branches        116      122       +6     
============================================
+ Hits           1364     1398      +34     
- Misses           78       89      +11     
- Partials         39       42       +3     
Flag Coverage Δ
ml-commons 91.43% <62.06%> (-0.67%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...h/ml/action/models/DeleteModelTransportAction.java 100.00% <ø> (ø)
...arch/ml/action/models/GetModelTransportAction.java 78.12% <ø> (ø)
...rch/ml/action/tasks/DeleteTaskTransportAction.java 100.00% <ø> (ø)
...search/ml/action/tasks/GetTaskTransportAction.java 78.12% <ø> (ø)
...rg/opensearch/ml/plugin/MachineLearningPlugin.java 100.00% <ø> (ø)
...rg/opensearch/ml/rest/RestMLSearchModelAction.java 100.00% <ø> (ø)
...org/opensearch/ml/rest/RestMLSearchTaskAction.java 100.00% <ø> (ø)
...java/org/opensearch/ml/rest/RestMLStatsAction.java 94.94% <ø> (ø)
...va/org/opensearch/ml/task/MLPredictTaskRunner.java 94.28% <ø> (ø)
...ain/java/org/opensearch/ml/task/MLTaskManager.java 87.61% <ø> (ø)
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us.

Signed-off-by: Yaliang Wu <ylwu@amazon.com>
} else {
listener.onFailure(new MLException("Failed to update index: " + indexName));
}
}, exception -> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we set indexMappingUpdated to false explicitly when getting an exception? I am not sure if there is a chance that we get an exception after setting indexMappingUpdated to true. So I recommend set to false here.

Copy link
Collaborator Author

@ylwu-amzn ylwu-amzn Aug 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The indexMappingUpdated flag is to make sure we only update the index mapping for first request. indexMappingUpdated was initialized as false, check line 44-45. We will set the flag as true if

  1. Index doesn't exist. When create index, it will use latest mapping.
  2. Index exists but its mapping is already latest (for example, some other request/node already updated the index mapping), then no need to update it.
  3. Index mapping is old, and we updated its mapping successfully.

For case 1 and 2, they map to line 104-105.
For case3, it maps to line 92.

When we set indexMappingUpdated as true, that means the index mapping already updated. No matter what happened later, no need to set indexMappingUpdated as false and update mapping again.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the explanation.

listener.onResponse(true);
}
}, e -> {
log.error("Failed to update index mapping", e);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

Copy link
Collaborator

@jngz-es jngz-es left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

@ylwu-amzn ylwu-amzn merged commit b9036b1 into opensearch-project:main Aug 4, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 4, 2022
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
(cherry picked from commit b9036b1)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 4, 2022
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
(cherry picked from commit b9036b1)
jngz-es pushed a commit that referenced this pull request Aug 4, 2022
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
(cherry picked from commit b9036b1)

Co-authored-by: Yaliang Wu <ylwu@amazon.com>
jngz-es pushed a commit that referenced this pull request Aug 4, 2022
Signed-off-by: Yaliang Wu <ylwu@amazon.com>
(cherry picked from commit b9036b1)

Co-authored-by: Yaliang Wu <ylwu@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants