Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update BWC zip links #625

Merged
merged 8 commits into from
Aug 4, 2022
Merged

Update BWC zip links #625

merged 8 commits into from
Aug 4, 2022

Conversation

ohltyler
Copy link
Member

@ohltyler ohltyler commented Aug 3, 2022

Signed-off-by: Tyler Ohlsen ohltyler@amazon.com

Description

Update BWC links to point to OpenSearch 1.1.0, since Open Distro is now officially unsupported. See OpenSearch maintenance policy on supported versions.

This PR also cleans up BWC tests to be compatible with 1.1 as the baseline. Note there was some changes to historical analysis between ODFE 1.13.0.0 (based on ES 7.10.2) => OpenSearch 1.1.0.0.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@ohltyler ohltyler requested a review from a team August 3, 2022 17:21
@opensearch-trigger-bot opensearch-trigger-bot bot added backport 2.x infra Changes to infrastructure, testing, CI/CD, pipelines, etc. labels Aug 3, 2022
@ohltyler ohltyler mentioned this pull request Aug 3, 2022
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@kaituo
Copy link
Collaborator

kaituo commented Aug 3, 2022

We have similar issue as in opensearch-project/OpenSearch#3001

Might want to try to use 1.0.1 instead of 1.0.0

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@ohltyler
Copy link
Member Author

ohltyler commented Aug 3, 2022

We have similar issue as in opensearch-project/OpenSearch#3001

Might want to try to use 1.0.1 instead of 1.0.0

Hmm, seems 1.0.1 is not published:

Could not determine the dependencies of task ':adBwcCluster#fullRestartClusterTask'.
> Can't get https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/1.0.1/latest/linux/x64/tar/builds/opensearch/plugins/opensearch-anomaly-detection-1.0.1.0.zip to /local/home/ohltyler/workspace/2.2/anomaly-detection-2/src/test/resources/org/opensearch/ad/bwc/anomaly-detection/1.0.1.0/opensearch-anomaly-detection-1.0.1.0.zip

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@ohltyler
Copy link
Member Author

ohltyler commented Aug 3, 2022

Since there is no components on version 1.0.1.x besides the security patched zip, including OpenSearch / OpenSearch Dashboards, it's inaccessible. Confirming if we should move to 1.1.0, or find a workaround to get the patched security zip.

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@ohltyler
Copy link
Member Author

ohltyler commented Aug 3, 2022

This will fail as 1.1 zips are not available at that link yet. Will re-run once the decision to move to 1.1 is finalized and zips are available.

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@ohltyler ohltyler changed the title Update BWC zip links Bump to 2.2.0.0; update BWC zip links Aug 4, 2022
@ohltyler ohltyler changed the title Bump to 2.2.0.0; update BWC zip links Update BWC zip links Aug 4, 2022
@ohltyler
Copy link
Member Author

ohltyler commented Aug 4, 2022

Hhistorical detector creation is failing for BWC, still investigating. Likely it's some of the historical detector refactoring changes between ODFE 1.13/OpenSearch 1.0.0 -> OpenSearch 1.1.0.

Suite: Test class org.opensearch.ad.bwc.ADBackwardsCompatibilityIT
  2> REPRODUCE WITH: ./gradlew ':adBwcCluster#oldVersionClusterTask1' --tests "org.opensearch.ad.bwc.ADBackwardsCompatibilityIT.testBackwardsCompatibility" -Dtests.seed=6734897D67D83C3A -Dtests.security.manager=false -Dtests.locale=lv -Dtests.timezone=America/Edmonton -Druntime.java=17
  2> java.lang.AssertionError: expected:<1> but was:<0>
        at __randomizedtesting.SeedInfo.seed([6734897D67D83C3A:8C0BB4B31D03E9B0]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.failNotEquals(Assert.java:835)
        at org.junit.Assert.assertEquals(Assert.java:6[47](https://github.com/opensearch-project/anomaly-detection/runs/7663704097?check_suite_focus=true#step:11:48))
        at org.junit.Assert.assertEquals(Assert.java:633)
        at org.opensearch.ad.bwc.ADBackwardsCompatibilityIT.createHistoricalAnomalyDetectorsAndStart(ADBackwardsCompatibilityIT.java:427)
        at org.opensearch.ad.bwc.ADBackwardsCompatibilityIT.testBackwardsCompatibility(ADBackwardsCompatibilityIT.java:156)

Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
@codecov-commenter
Copy link

codecov-commenter commented Aug 4, 2022

Codecov Report

Merging #625 (6d2183f) into main (f630c8f) will decrease coverage by 0.08%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##               main     #625      +/-   ##
============================================
- Coverage     79.07%   78.99%   -0.09%     
- Complexity     4213     4234      +21     
============================================
  Files           296      301       +5     
  Lines         17686    17869     +183     
  Branches       1880     1894      +14     
============================================
+ Hits          13986    14116     +130     
- Misses         2805     2847      +42     
- Partials        895      906      +11     
Flag Coverage Δ
plugin 78.99% <ø> (-0.09%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...search/ad/cluster/ClusterManagerEventListener.java 78.72% <0.00%> (-15.88%) ⬇️
.../java/org/opensearch/ad/AnomalyDetectorRunner.java 37.64% <0.00%> (-5.89%) ⬇️
.../org/opensearch/ad/ratelimit/ColdEntityWorker.java 90.90% <0.00%> (-4.55%) ⬇️
...ansport/handler/AnomalyResultBulkIndexHandler.java 83.87% <0.00%> (-3.23%) ⬇️
.../org/opensearch/ad/feature/CompositeRetriever.java 81.45% <0.00%> (-3.04%) ⬇️
...opensearch/ad/ratelimit/CheckpointWriteWorker.java 91.25% <0.00%> (-2.58%) ⬇️
...in/java/org/opensearch/ad/caching/CacheBuffer.java 77.69% <0.00%> (-1.17%) ⬇️
...nsearch/ad/transport/RCFResultTransportAction.java 85.18% <0.00%> (-0.87%) ⬇️
.../java/org/opensearch/ad/caching/PriorityCache.java 69.38% <0.00%> (-0.87%) ⬇️
.../opensearch/ad/ratelimit/CheckpointReadWorker.java 87.01% <0.00%> (-0.67%) ⬇️
... and 19 more

@ohltyler ohltyler merged commit 7f84601 into opensearch-project:main Aug 4, 2022
@ohltyler ohltyler deleted the fix-bwc branch August 4, 2022 23:46
@opensearch-trigger-bot
Copy link

The backport to 1.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.x 1.x
# Navigate to the new working tree
cd .worktrees/backport-1.x
# Create a new branch
git switch --create backport/backport-625-to-1.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7f846016c92800383ed848936ed82251108c8947
# Push it to GitHub
git push --set-upstream origin backport/backport-625-to-1.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.x

Then, create a pull request where the base branch is 1.x and the compare/head branch is backport/backport-625-to-1.x.

opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 4, 2022
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
(cherry picked from commit 7f84601)
ohltyler added a commit that referenced this pull request Aug 5, 2022
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
(cherry picked from commit 7f84601)
kaituo pushed a commit to kaituo/anomaly-detection-1 that referenced this pull request Nov 12, 2022
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
kaituo pushed a commit to kaituo/anomaly-detection-1 that referenced this pull request Nov 15, 2022
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
kaituo pushed a commit to kaituo/anomaly-detection-1 that referenced this pull request Nov 15, 2022
Signed-off-by: Tyler Ohlsen <ohltyler@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.x backport 2.x infra Changes to infrastructure, testing, CI/CD, pipelines, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants