Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests fail in a project with splashscreen #733

Open
RedX2501 opened this issue Aug 21, 2024 · 14 comments
Open

Tests fail in a project with splashscreen #733

RedX2501 opened this issue Aug 21, 2024 · 14 comments
Labels
help wanted Extra attention is needed investigation required
Milestone
7.x.y

Comments

@RedX2501
Copy link

I have a project that runs fine with Webdriver v8.40.3 and wdie-electron-service 6.

I'm trying to migrate to v9.

I have tried using 7.0.0-beta.10 but that did not solve the problem.

I have created a fresh wdio test project to make sure it's not a cnofiguration error.

I'm using electron 29.1.4.

describe('Electron Testing', () => {
    it('should print application title', async () => {
        await browser.pause(10_000); // Added so that the main window has time to load.
        console.log('Hello', await browser.getTitle(), 'application!')
    })
})

Error:

[0-0] 2024-08-20T06:51:03.421Z INFO webdriver: COMMAND getWindowHandles()
[0-0] 2024-08-20T06:51:03.421Z INFO webdriver: [GET] http://localhost:55479/session/d39baa3e59c3e5c9c8035d7328329e58/window/handles
[0-0] 2024-08-20T06:51:03.424Z INFO webdriver: RESULT [ '3F1FAD1B8DFC8444955D7546C4B2D76B' ]
[0-0] 2024-08-20T06:51:03.425Z INFO HtmlReporter: onRunnerStart: 0-0
[0-0] 2024-08-20T06:51:03.427Z INFO HtmlReporter: onHookStart: 0-0:hook-0-0
[0-0] 2024-08-20T06:51:03.427Z INFO HtmlReporter: onHookEnd: 0-0:hook-0-0
[0-0] 2024-08-20T06:51:03.428Z INFO HtmlReporter: onSuiteStart: 0-0:suite-0-0
[0-0] 2024-08-20T06:51:03.428Z INFO HtmlReporter: onTestStart: 0-0:test-00-0
[0-0] 2024-08-20T06:51:13.437Z INFO webdriver: COMMAND getTitle()
[0-0] 2024-08-20T06:51:13.437Z INFO webdriver: [GET] http://localhost:55479/session/d39baa3e59c3e5c9c8035d7328329e58/title
[0-0] 2024-08-20T06:51:13.442Z WARN webdriver: Request failed with status 404 due to no such window: target window already closed       
[0-0] from unknown error: web view not found
[0-0]   (Session info: chrome=122.0.6261.129)
[0-0] 2024-08-20T06:51:13.442Z INFO webdriver: Retrying 1/3
[0-0] 2024-08-20T06:51:13.442Z INFO webdriver: [GET] http://localhost:55479/session/d39baa3e59c3e5c9c8035d7328329e58/title
[0-0] 2024-08-20T06:51:13.444Z WARN webdriver: Request failed with status 404 due to no such window: target window already closed       
[0-0] from unknown error: web view not found
[0-0]   (Session info: chrome=122.0.6261.129)
[0-0] 2024-08-20T06:51:13.444Z INFO webdriver: Retrying 2/3
[0-0] 2024-08-20T06:51:13.444Z INFO webdriver: [GET] http://localhost:55479/session/d39baa3e59c3e5c9c8035d7328329e58/title
[0-0] 2024-08-20T06:51:13.446Z WARN webdriver: Request failed with status 404 due to no such window: target window already closed       
[0-0] from unknown error: web view not found
[0-0]   (Session info: chrome=122.0.6261.129)
[0-0] 2024-08-20T06:51:13.446Z INFO webdriver: Retrying 3/3
[0-0] 2024-08-20T06:51:13.446Z INFO webdriver: [GET] http://localhost:55479/session/d39baa3e59c3e5c9c8035d7328329e58/title
[0-0] 2024-08-20T06:51:13.447Z ERROR webdriver: Request failed with status 404 due to no such window: no such window: target window already closed
[0-0] from unknown error: web view not found
[0-0]   (Session info: chrome=122.0.6261.129)
[0-0] no such window in "Electron Testing.should print application title"
no such window: no such window: target window already closed
from unknown error: web view not found
  (Session info: chrome=122.0.6261.129)
    at Context.<anonymous> (c:\data\_eev\testproj\test\specs\test.e2e.ts:6:30)
[0-0] 2024-08-20T06:51:13.448Z INFO HtmlReporter: onTestFail: 0-0:test-00-0
[0-0] 2024-08-20T06:51:13.449Z INFO HtmlReporter: onTestEnd: 0-0:test-00-0
[0-0] 2024-08-20T06:51:13.449Z INFO HtmlReporter: onSuiteEnd: 0-0:suite-0-0
[0-0] 2024-08-20T06:51:13.449Z INFO HtmlReporter: onHookStart: 0-0:hook-0-1
[0-0] 2024-08-20T06:51:13.449Z INFO HtmlReporter: onHookEnd: 0-0:hook-0-1
[0-0] 2024-08-20T06:51:13.449Z INFO webdriver: COMMAND deleteSession()
[0-0] 2024-08-20T06:51:13.450Z INFO webdriver: [DELETE] http://localhost:55479/session/d39baa3e59c3e5c9c8035d7328329e58
[0-0] 2024-08-20T06:51:13.486Z INFO webdriver: RESULT null
[0-0] 2024-08-20T06:51:13.486Z INFO webdriver: Kill driver process with PID 36272
[0-0] 2024-08-20T06:51:13.488Z INFO HtmlReporter: onRunnerEnd: 0-0
[0-0] 2024-08-20T06:51:13.488Z INFO HtmlReporter: onRunnerEnd: 0-0 awaiting report generation
[0-0] 2024-08-20T06:51:13.488Z INFO ReportGenerator: Report Generation started
[0-0] 2024-08-20T06:51:13.488Z INFO ReportGenerator: Included metrics for suite: 0-0 suite-0-0
[0-0] 2024-08-20T06:51:13.489Z INFO ReportGenerator: Generated 1 specs, 1 suites,
[0-0] 2024-08-20T06:51:13.489Z INFO JsonGenerator: Json stringify starting: C:\data\_eev\testproj\reports\html-reports\report-0-0.json  
[0-0] 2024-08-20T06:51:13.501Z INFO JsonGenerator: Json stringify complete
[0-0] 2024-08-20T06:51:13.502Z INFO JsonGenerator: Json write completed: C:\data\_eev\testproj\reports\html-reports\report-0-0.json     
[0-0] 2024-08-20T06:51:13.503Z INFO HtmlGenerator: Html Generation started
[0-0] 2024-08-20T06:51:13.503Z INFO HtmlGenerator: Html Generation started in C:\data\_eev\testproj\node_modules\wdio-html-nice-reporter[0-0] 2024-08-20T06:51:13.516Z INFO HtmlGenerator: Html Generation Completed
[0-0] 2024-08-20T06:51:13.517Z INFO ReportGenerator: Report Generation completed
[0-0] 2024-08-20T06:51:13.593Z INFO HtmlReporter: isSynchronized: true
[0-0] FAILED in chrome - file:///C:/data/_eev/testproj/test/specs/test.e2e.ts
2024-08-20T06:51:13.606Z INFO @wdio/cli:launcher: Run onWorkerEnd hook
2024-08-20T06:51:13.606Z INFO @wdio/cli:launcher: Run onComplete hook

 "spec" Reporter:
------------------------------------------------------------------
[chrome 122.0.6261.129 windows #0-0] Running: chrome (v122.0.6261.129) on windows
[chrome 122.0.6261.129 windows #0-0] Session ID: d39baa3e59c3e5c9c8035d7328329e58
[chrome 122.0.6261.129 windows #0-0]
[chrome 122.0.6261.129 windows #0-0] » \test\specs\test.e2e.ts
[chrome 122.0.6261.129 windows #0-0] Electron Testing
[chrome 122.0.6261.129 windows #0-0]    ✖ should print application title
[chrome 122.0.6261.129 windows #0-0]
[chrome 122.0.6261.129 windows #0-0] 1 failing (10s)
[chrome 122.0.6261.129 windows #0-0]
[chrome 122.0.6261.129 windows #0-0] 1) Electron Testing should print application title
[chrome 122.0.6261.129 windows #0-0] no such window: target window already closed
from unknown error: web view not found
  (Session info: chrome=122.0.6261.129)
[chrome 122.0.6261.129 windows #0-0] no such window: no such window: target window already closed
[chrome 122.0.6261.129 windows #0-0] from unknown error: web view not found
[chrome 122.0.6261.129 windows #0-0]   (Session info: chrome=122.0.6261.129)
[chrome 122.0.6261.129 windows #0-0]     at Context.<anonymous> (c:\data\_eev\testproj\test\specs\test.e2e.ts:6:30)


Spec Files:      0 passed, 1 failed, 1 total (100% completed) in 00:00:14

The project uses a splash screen that closes once the main window successfully connects to the backend.
@goosewobbler
Copy link
Member

goosewobbler commented Aug 21, 2024 •
edited
Loading

There's been some issues getting the next major out, see #533 for details. The betas have been tested with WDIO v9 in wdio-electron-service-example and are working there.

It sounds like Chromedriver is trying to connect to the splash screen. I'm not sure why this would change with WDIO v9...

@goosewobbler goosewobbler added question Further information is requested investigation required labels Aug 21, 2024
@goosewobbler
Copy link
Member

goosewobbler commented Aug 21, 2024 •
edited
Loading

v7 of the service has been released now, this only works with WDIO v9. Can you try it again with the latest builds? Presumably the tests work when the splash screen is disabled?

@RedX2501
Copy link
Author

How does chromedriver know whether somthing is a splash screen or de main window?

@goosewobbler
Copy link
Member

Do you have a repo to look at? It's not easy to debug without seeing what the app is actually doing.

@RedX2501
Copy link
Author

RedX2501 commented Aug 22, 2024 •
edited
Loading

I have now :) See RedX2501/electron-react-test@c30f1f3

I have a separate project in which the tests are. I will package the electron app and then use appBinaryPath to test. This setup is necessary as the tool is developed by a third-party and we receive the source code and binaries on release.

I write acceptance tests for the binaries we receive to make sure they comply with our specs.

@RedX2501
Copy link
Author

RedX2501 commented Aug 22, 2024 •
edited
Loading

I have tried with the released v7 and it's still the same problem.

If I change the code to find all the handles and then switch to the only handle available it actually works.

This behaviour I know from the v6 version, although I think it's counter intuitive.

I need to enumerate all handles because calling browser.getHandle() actually throws when the first window (splash) has closed and only the second windows is showing.

The error is

no such window: target window already closed
from unknown error: web view not found
  (Session info: chrome=122.0.6261.129)

@goosewobbler
Copy link
Member

goosewobbler commented Aug 22, 2024 •
edited
Loading

So, does it work if you disable the splash screen?

Also, you mention the approach of changing the code to switch handles was known from the previous version. Does this mean the error also occurs in WDIO v8 if you do not do this step?

I'm just trying to properly define this issue as it is clear the splash screen use case should be supported, but it's not obvious how this relates to the new v9 release (if it does at all).

I'm going to refer only to WDIO versions going forward, you should be using v7.x of the service with v9.x of WDIO, and v6.x of the service with v8.x of WDIO.

@RedX2501
Copy link
Author

Hi,

I have tested it some more.

If I have only one window it does work.
So having only the Splashscreen and never opening MainWindow works fine.
Having only MainWindow and never opening the Splashscreen also works.
Having Splashscreen stay open long enough for the test to run to completion also works.

So basically it boils down to my expectation that browser.getTitle() should just work if there is one window only.

I'm fine with closing this ticket as wrong expectation.

Should I open another one about the getTitle (and other functions) failing when the first window closes? In this repo or another? Is this behaviour documented somewhere?

@goosewobbler goosewobbler changed the title Simple tests fail with WebdriverIO 9.0 Tests fail in a project with splashscreen Aug 27, 2024
@goosewobbler
Copy link
Member

goosewobbler commented Aug 27, 2024 •
edited
Loading

So it definitely sounds like the display time of the splash is too short for WDIO to run tests on the splash, which results in the target window closed error.

I would have a separate splashscreen.spec.ts file where you run tests which assert properties of the splash, you might also assert that it disappears after a certain time, for this spec you will likely need to operate your splash in a "test mode" where you can customise the display time, how long the splash is displayed before it disappears.

Then all other spec files could either wait until the splash is replaced with the main window, or simply run the app with a splash display time of 0 - so the splash is replaced with the main window immediately - before running their tests.

You could search (or ask) in the main WDIO forum about how best to test projects with splash screens, I'm sure there will have been others with the same issue. There is also the WDIO Discord where you can ask questions around testing with WDIO.

We could consider some kind of splash screen helper in the service but I'm not sure it would save much time or code, the solution here seems to be an improved structure for your tests to test the splash separately from the main window.

@goosewobbler goosewobbler closed this as not planned Won't fix, can't repro, duplicate, stale Aug 27, 2024
@RedX2501
Copy link
Author

I think i didn't express myself correctly.

I don't want to test the splashscreen, what's undocumented/weird ist that even if i wait long enough for the main window to appear browser.getTitle fails.

Only after switchToWindow it works again.

I would have expected to not need to explicitly call switchToWindow when getHandles returns only one entry.

@goosewobbler
Copy link
Member

Ok, I see. That sounds like something which could have a solution attached. So you're saying that you need to explicitly call switchToWindow after the splash is closed, in order to switch to the main window from the splash.

@christian-bromann Do you think there's a feature request here? It would make sense if WDIO could auto-switch windows in this splash screen case.

@goosewobbler goosewobbler reopened this Aug 27, 2024
@RedX2501
Copy link
Author

RedX2501 commented Aug 28, 2024 •
edited
Loading

This affects many other functions too even getHandle (without s). It won't retrieve the current handle of the single window if one window closed and another opened. It only starts to work after switchToWindow.

@goosewobbler
Copy link
Member

Possibly related: webdriverio/webdriverio#13436

@goosewobbler goosewobbler removed the question Further information is requested label Aug 28, 2024
@christian-bromann
Copy link
Contributor

Do you think there's a feature request here? It would make sense if WDIO could auto-switch windows in this splash screen case.

I would assume that the WebDriver protocol automatically switches the window handle if a window closes. Let me verify that.

@goosewobbler goosewobbler added the help wanted Extra attention is needed label Sep 9, 2024
@goosewobbler goosewobbler added this to the 7.x.y milestone Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Extra attention is needed investigation required
Projects
None yet
Milestone
7.x.y
Development

No branches or pull requests
3 participants
@goosewobbler @christian-bromann @RedX2501