Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ACTION NEEDED] Fix flaky integration tests at distribution level #1362

Closed
Tracked by #4588
gaiksaya opened this issue Apr 3, 2024 · 4 comments
Closed
Tracked by #4588

[ACTION NEEDED] Fix flaky integration tests at distribution level #1362

gaiksaya opened this issue Apr 3, 2024 · 4 comments
Labels
bug Something isn't working untriaged v2.14.0 Issues targeting release v2.14.0

Comments

@gaiksaya
Copy link
Member

gaiksaya commented Apr 3, 2024

What is the bug?
It was observed in 2.13.0 and previous other releases that this component manually signed off on the release for failing integration tests. See opensearch-project/opensearch-build#4433 (comment)
The flakiness of the test runs take a lot of time from the release team to collect go/no-go decision and significantly lower the confidence in the release bundles.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Run integration testing for altering and see the failures.
  2. Issues can be reproduced using the steps declared in AUTOCUT issues for failed integration testing

What is the expected behavior?
Tests should be consistently passing.

Do you have any additional context?
Please note that this is a hard blocker for 2.14.0 release as per the discussion here

@gaiksaya gaiksaya added bug Something isn't working untriaged v2.14.0 Issues targeting release v2.14.0 labels Apr 3, 2024
@ankitkala
Copy link
Member

Related issue: opensearch-project/opensearch-build#4610

@bbarani
Copy link
Member

bbarani commented Apr 23, 2024

FYI... We will ignore the failing test for Deb / RPM in 2.14.0 release if we cannot come up with a solution before the release timeline. We should close this gap for 2.15.0 release though. CC: @rishabh6788 @gaiksaya @peterzhuamazon

@gaiksaya
Copy link
Member Author

Adding 2.14.0 release manager @rishabh6788

@dblock
Copy link
Member

dblock commented Jun 24, 2024

In 2.15 we ran with automation end-to-end, closing.

Catch All Triage - 1 2 3 4 5 6

@dblock dblock closed this as completed Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working untriaged v2.14.0 Issues targeting release v2.14.0
Projects
None yet
Development

No branches or pull requests

4 participants