chore(testrunner): distinguish between TERMINATED and CRASHED #4821
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
`testRunner.run()` might have 4 different outcomes:
- `ok` - all non-skipped tests passed
- `failed` - some tests failed or timed out
- `terminated` - process received SIGHUP/SIGINT while testrunner was running tests. This happens on CI's under certain circumstances, e.g. when
VM is getting re-scheduled.
- `crashed` - testrunner terminated test execution due to either `UnhandledPromiseRejection` or
some of the hooks (`beforeEach/afterEach/beforeAll/afterAll`) failures.
As an implication, there are 2 new test results: `terminated` and `crashed`.
All possible test results are:
- `ok` - test worked just fine
- `skipped` - test was skipped with `xit`
- `timedout` - test timed out
- `failed` - test threw an exception while running
- `terminated` - testrunner got terminated while running this test
- `crashed` - some `beforeEach` / `afterEach` hook corresponding to this
test timed out of threw an exception.
This patch changes a few parts of the testrunner API:
- `testRunner.run()` now returns an object `{result: string,
terminationError?: Error, terminationMessage?: string}`
- the same object is dispatched via `testRunner.on('finished')` event
- `testRunner.on('terminated')` got removed
- tests now might have `crashed` and `terminated` results
- `testRunner.on('teststarted')` dispatched before running all related
`beforeEach` hooks, and `testRunner.on('testfinished')` dispatched after
running all related `afterEach` hooks.
JoelEinbinder
approved these changes
Aug 8, 2019
This was referenced Dec 4, 2019
rfojtik
pushed a commit
to rfojtik/puppeteer
that referenced
this pull request
Dec 21, 2019
…eer#4821) `testRunner.run()` might have 4 different outcomes: - `ok` - all non-skipped tests passed - `failed` - some tests failed or timed out - `terminated` - process received SIGHUP/SIGINT while testrunner was running tests. This happens on CI's under certain circumstances, e.g. when VM is getting re-scheduled. - `crashed` - testrunner terminated test execution due to either `UnhandledPromiseRejection` or some of the hooks (`beforeEach/afterEach/beforeAll/afterAll`) failures. As an implication, there are 2 new test results: `terminated` and `crashed`. All possible test results are: - `ok` - test worked just fine - `skipped` - test was skipped with `xit` - `timedout` - test timed out - `failed` - test threw an exception while running - `terminated` - testrunner got terminated while running this test - `crashed` - some `beforeEach` / `afterEach` hook corresponding to this test timed out of threw an exception. This patch changes a few parts of the testrunner API: - `testRunner.run()` now returns an object `{result: string, terminationError?: Error, terminationMessage?: string}` - the same object is dispatched via `testRunner.on('finished')` event - `testRunner.on('terminated')` got removed - tests now might have `crashed` and `terminated` results - `testRunner.on('teststarted')` dispatched before running all related `beforeEach` hooks, and `testRunner.on('testfinished')` dispatched after running all related `afterEach` hooks.
This was referenced Feb 3, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
testRunner.run()might have 4 different outcomes:ok- all non-skipped tests passedfailed- some tests failed or timed outterminated- process received SIGHUP/SIGINT while testrunner was running tests. This happens on CI's under certain circumstances, e.g. whenVM is getting re-scheduled.
crashed- testrunner terminated test execution due to eitherUnhandledPromiseRejectionorsome of the hooks (
beforeEach/afterEach/beforeAll/afterAll) failures.As an implication, there are 2 new test results:
terminatedandcrashed.All possible test results are:
ok- test worked just fineskipped- test was skipped withxittimedout- test timed outfailed- test threw an exception while runningterminated- testrunner got terminated while running this testcrashed- somebeforeEach/afterEachhook corresponding to thistest timed out of threw an exception.
This patch changes a few parts of the testrunner API:
testRunner.run()now returns an object{result: string, terminationError?: Error, terminationMessage?: string}testRunner.on('finished')eventtestRunner.on('terminated')got removedcrashedandterminatedresultstestRunner.on('teststarted')dispatched before running all relatedbeforeEachhooks, andtestRunner.on('testfinished')dispatched afterrunning all related
afterEachhooks.