Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing test: Chrome X-Pack UI Functional Tests Basic License.x-pack/test/functional/apps/ml/data_visualizer/index_data_visualizer·ts - apps machine learning basic license data visualizer index based with farequote lucene saved search displays index details #105087

Closed
kibanamachine opened this issue Jul 9, 2021 · 9 comments
Closed

Failing test: Chrome X-Pack UI Functional Tests Basic License.x-pack/test/functional/apps/ml/data_visualizer/index_data_visualizer·ts - apps machine learning basic license data visualizer index based with farequote lucene saved search displays index details #105087

kibanamachine opened this issue Jul 9, 2021 · 9 comments
Assignees
@pheyos
Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test :ml skipped-test v7.15.0

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Jul 9, 2021 •
edited
Loading

A test failed on a tracked branch

Error: retry.tryForTime timeout: Error: Expected total document count to be '34,416' (got '86,274')
    at Assertion.assert (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:100:11)
    at Assertion.eql (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/expect/expect.js:244:8)
    at /dev/shm/workspace/parallel/12/kibana/x-pack/test/functional/services/ml/data_visualizer_index_based.ts:28:29
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at runAttempt (/dev/shm/workspace/parallel/12/kibana/test/common/services/retry/retry_for_success.ts:27:15)
    at retryForSuccess (/dev/shm/workspace/parallel/12/kibana/test/common/services/retry/retry_for_success.ts:66:21)
    at RetryService.tryForTime (/dev/shm/workspace/parallel/12/kibana/test/common/services/retry/retry.ts:22:12)
    at Object.assertTotalDocumentCount (test/functional/services/ml/data_visualizer_index_based.ts:26:7)
    at Object.clickUseFullDataButton (test/functional/services/ml/data_visualizer_index_based.ts:37:7)
    at Context.<anonymous> (test/functional/apps/ml/data_visualizer/index_data_visualizer.ts:397:7)
    at Object.apply (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)
    at onFailure (/dev/shm/workspace/parallel/12/kibana/test/common/services/retry/retry_for_success.ts:17:9)
    at retryForSuccess (/dev/shm/workspace/parallel/12/kibana/test/common/services/retry/retry_for_success.ts:57:13)
    at RetryService.tryForTime (/dev/shm/workspace/parallel/12/kibana/test/common/services/retry/retry.ts:22:12)
    at Object.assertTotalDocumentCount (test/functional/services/ml/data_visualizer_index_based.ts:26:7)
    at Object.clickUseFullDataButton (test/functional/services/ml/data_visualizer_index_based.ts:37:7)
    at Context.<anonymous> (test/functional/apps/ml/data_visualizer/index_data_visualizer.ts:397:7)
    at Object.apply (/dev/shm/workspace/parallel/12/kibana/node_modules/@kbn/test/target_node/functional_test_runner/lib/mocha/wrap_function.js:87:16)

First failure: Jenkins Build
@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Jul 9, 2021
@botelastic botelastic bot added the needs-team Issues missing a team label label Jul 9, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/ml-ui (:ml)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Jul 9, 2021
@spalger
Copy link
Contributor

spalger commented Jul 9, 2021

This test is not very flaky, though it has failed 4 times in the last 30 days and three of those times have been in the last 2 days, all on master and PRs to master:

image

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@pheyos pheyos self-assigned this Jul 12, 2021
@pheyos
Copy link
Member

pheyos commented Jul 12, 2021

At glance this looks like sometimes the saved search isn't applied correctly. I'll take a closer look.

This was referenced Jul 12, 2021
Closed
Closed
@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

@kibanamachine
Copy link
Contributor Author

New failure: Jenkins Build

spalger added a commit that referenced this issue Jul 15, 2021
@spalger
skip flaky suite (#105087)
961681f
spalger added a commit that referenced this issue Jul 15, 2021
@spalger
skip flaky suite (#105087)
5f5f097 
(cherry picked from commit 961681f)
@spalger
Copy link
Contributor

spalger commented Jul 15, 2021

Thanks for taking a look @pheyos, and it seems like you were able to reproduce this in #105204 and it keeps happening, so skipping for now

master/8.0: 961681f
7.x/7.15: 5f5f097

jloleysens added a commit to jloleysens/kibana that referenced this issue Jul 15, 2021
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into reporting/new…
cec3989 
…-png-pdf-report-type

* 'master' of github.com:elastic/kibana: (75 commits)
  [Search Sessions] Don’t try to delete errored searches (elastic#105434)
  [Search Sessions] fix saved object can be created even if courier:batchSearches is enabled (elastic#105407)
  [Remote Clusters] Fixed remote clusters details flyout for long strings (elastic#105592)
  [ML] Functional tests - re-activate a11y tests (elastic#105198)
  [APM] Typed client-side routing (elastic#104274)
  [Canvas] Expression error (elastic#103048)
  [ML] Fixing job wizard with missing description (elastic#105574)
  [Security Solution][Alerts] - Add alerts subfeature UI (elastic#105505)
  Upgrade EUI to v35.0.0 (elastic#105127)
  [Reporting] Clean up types for internal APIs needed for UI (elastic#105508)
  skip flaky suite (elastic#105087)
  [Workplace Search] Fix Chrome issues with GitHub sources (elastic#105680)
  [Fleet] Add containerized fleet server instructions to Fleet README (elastic#105669)
  [ML] Add api integration test for analytics map endpoint  (elastic#105531)
  Fixes cypress flake across two tests (elastic#105645)
  [Logs&Metrics UI] add owner properties to plugin manifest (elastic#105580)
  chore(NA): introduce preset for jest-integration tests on @kbn/test (elastic#105144)
  [Enterprise Search] Added Thumbnails to Search UI (elastic#104199)
  Translate App Search credentials list (elastic#105619)
  [APM] APM agent config created prior to Fleet migration is not injected into integration policy (elastic#105504)
  ...

# Conflicts:
#	x-pack/plugins/reporting/common/types.ts
#	x-pack/plugins/reporting/public/management/report_listing.test.tsx
@pheyos
Copy link
Member

pheyos commented Jul 16, 2021

Thanks for skipping the flaky tests for now @spalger . Unfortunately, I wasn't able to reproduce this with the flaky test runner so far in 250 runs, but will continue to investigate.

This was referenced Jul 16, 2021
Closed
Closed
@pheyos
Copy link
Member

pheyos commented Jul 20, 2021

The race condition has been fixed and tests have been re-enabled in #106007.

@pheyos pheyos closed this as completed Jul 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pheyos pheyos

Labels
blocker failed-test A test failure on a tracked branch, potentially flaky-test :ml skipped-test v7.15.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[ML] Functional tests - stabilize index data visualizer tests pheyos/kibana
5 participants
@spalger @pheyos @peteharverson @elasticmachine @kibanamachine