Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] Make FTR tests independent #1167

Open
kavilla opened this issue Jan 20, 2022 · 0 comments
Open

[Improvement] Make FTR tests independent #1167

kavilla opened this issue Jan 20, 2022 · 0 comments
Labels
enhancement New feature or request technical debt If not paid, jeapardizes long-term success and maintainability of the repository. test:functional

Comments

@kavilla
Copy link
Member

kavilla commented Jan 20, 2022

Description

Executing yarn test:ftrcould potentially lead to failures (or unexpected consequences) in a series of unrelated functional tests because an earlier test failed (or even sometimes succeeded) and prevented the test from closing in a "zero state".

For example, test group 1 could have the following test files: TestSettings, TestOverviewPage, TestDiscoverPage. All the tests would require returning to the Home page in the "clean up" step and require to start in the Home page. If TestOveragePage ends up not rendering properly, the test runner can fail and not execute the "clean up" step thus not putting the application the Home page and TestDiscoverPage failed because it was trying to use the test runner to navigate to the DiscoverPage from the Home page.

That is for failures, but for non-failures, TestSettings will impact TestOverviewPage and TestDiscoverPage, and TestOverviewPage will impact TestDiscoverPage. This becomes problematic if we want to test the welcomeScreen because if the application is not in a "zero state" with no actual docs in the system index then it will not show that welcomeScreen. So if TestSettings adding docs to the system index then it would prevent a later test from seeing that welcomeScreen if there is a specific test for that. Putting the test first would still not be desired because we should be able to run any file without worrying about previous runs.

Expected Result

Functional tests are independent. If the test requires going to Settings and Overview page to test the Discover page then the Discover test should execute as a setup step.

Pros

Tests are more isolated and granular to figure out the right component that failed. So one failure doesn't cause a failure in a suite of tests.

Cons

Longer execution time

@kavilla kavilla added enhancement New feature or request test:functional labels Jan 20, 2022
AMoo-Miki pushed a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this issue Feb 10, 2022
AMoo-Miki pushed a commit to AMoo-Miki/OpenSearch-Dashboards that referenced this issue Feb 10, 2022
# [30.0.0](elastic/elastic-charts@v29.2.0...v30.0.0) (2021-06-04)

### Bug Fixes

* **domain:** custom domain should not filter data ([opensearch-project#1181](elastic/elastic-charts#1181)) ([92ba84c](elastic/elastic-charts@92ba84c)), closes [opensearch-project#1129](elastic/elastic-charts#1129)
* **value_labels:** zero as a valid value for textBorder and borderWidth ([#1182](elastic/elastic-charts#1182)) ([880fbf1](elastic/elastic-charts@880fbf1))
* annotation tooltip display when remounting specs ([opensearch-project#1167](elastic/elastic-charts#1167)) ([7163951](elastic/elastic-charts@7163951))
* render nodeLabel formatted text into the nodes ([opensearch-project#1173](elastic/elastic-charts#1173)) ([0de9688](elastic/elastic-charts@0de9688))

### Features

* **axis:** allow pixel domain padding for y axes  ([#1145](elastic/elastic-charts#1145)) ([6787728](elastic/elastic-charts@6787728))
* apply value formatter to the default legend item label ([opensearch-project#1190](elastic/elastic-charts#1190)) ([20108bb](elastic/elastic-charts@20108bb))
* **tooltip:** stickTo vertical middle of the cursor ([#1163](elastic/elastic-charts#1163)) ([b858fb3](elastic/elastic-charts@b858fb3)), closes [opensearch-project#1108](elastic/elastic-charts#1108)
* **wordcloud:** click and over events on text ([opensearch-project#1180](elastic/elastic-charts#1180)) ([adbf341](elastic/elastic-charts@adbf341)), closes [opensearch-project#1156](elastic/elastic-charts#1156)

### BREAKING CHANGES

* **value_labels:** the `textBorder` of `ValueFillDefinition` is now optional or a number only
@tmarkley tmarkley added the technical debt If not paid, jeapardizes long-term success and maintainability of the repository. label May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request technical debt If not paid, jeapardizes long-term success and maintainability of the repository. test:functional
Projects
None yet
Development

No branches or pull requests

2 participants