Failing ES Promotion: FTR Configs #36 / dashboard Reporting Dashboard Reporting Screenshots Preserve Layout downloads a PDF file: large dashboard

[mistic]

Chrome X-Pack UI Functional Tests
x-pack/test/functional/apps/dashboard/group3/reporting/screenshots.ts

dashboard Reporting Dashboard Reporting Screenshots Preserve Layout downloads a PDF file: large dashboard

This failure is preventing the promotion of the current Elasticsearch nightly snapshot.

For more information on the Elasticsearch snapshot promotion process including how to reproduce using the unverified nightly ES build: https://www.elastic.co/guide/en/kibana/master/development-es-snapshots.html

• Failed promotion job
• Test Failure

TimeoutError: Waiting for element to be located By(css selector, [data-test-errorText])
Wait timed out after 10027ms
    at /var/lib/buildkite-agent/builds/kb-n2-4-spot-c0c3875bd7799fc1/elastic/kibana-elasticsearch-snapshot-verify/kibana/node_modules/selenium-webdriver/lib/webdriver.js:929:17
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5) {
  remoteStacktrace: ''
}

[elasticmachine]

Pinging @elastic/appex-sharedux (Team:SharedUX)

[mistic]

Skipped.

main: 8941058

[tsullivan]

This test is rightfully skipped for two reasons. ES promotion failures currently stop the rendered report from being stored: #157344

This test runs a "large" report, which currently has a separate known issue: #138812

[tsullivan]

Blocked on #138812

[rayafratkina]

Any update on this? It's been quiet for a while now

[tsullivan]

@rayafratkina this is blocked because currently we can't support screenshot reports of arbitrary size. The original test ignored the problems with it. This has been skipped for a long time but the work to support the needed functionality is on the roadmap.

I think there are a few measures we can take to make large screenshot testing better:

1. We may choose to delete these tests for now and bring them back once we build better support of capturing large dashboards.
2. Then we only need 1 test for a large PNG, not PDF.
3. The test should be decoupled from the Dashboards app. We can create an example app under SharedUx ownership to verify capturing a large screenshot report.

[rayafratkina]

If we are not planning to fix this test in the next sprint or two, I think deleting these tests until we can properly run them is a reasonable option. We should not leave skipped flaky tests sitting for long periods of time - they are just generating noise without contributing anything
cc @jb1b84 @ThomThomson

[ThomThomson]

Agreed that these tests should be deleted for the time being. Good plan @tsullivan!

[tsullivan]

Thanks for the helpful discussion. A PR is now staged for review: https://github.com/elastic/kibana/pull/163966/files

Cheers!