[antithesis] Enable reuse of banff e2e test for antithesis testing #3554
Conversation
marun
changed the title
|
Switched to draft in light of wanting to rebase on #3557 |
marun
force-pushed
the
testing-switch-to-zap
branch
from
cf8a4d4 to
292016a
Compare
|
Moving to draft and adding a bunch of TODOs prompted by a manually-triggered antithesis run. The error handling of the e2e test is not sufficiently robust, so there are a lot of unhelpful errors that need to be addressed before this PR will be mergeable. |
marun
force-pushed
the
e2e-reuse
branch
3 times, most recently
from
6e0b41c to
981663d
Compare
|
This PR has become stale because it has been open for 30 days with no activity. Adding the |
|
This PR has become stale because it has been open for 30 days with no activity. Adding the |
maru-ava
requested review from
Elvis339,
joshua-kim and
RodrigoVillar
and removed request for
StephenButtolph and
Elvis339
There was a problem hiding this comment.
Pull Request Overview
This PR refactors the Banff end-to-end tests to be reusable by both Ginkgo and Antithesis harnesses, enhancing context management and cleanup handling.
- Introduce
APITestFunctionandExecuteAPITestto unify API-based test execution - Extend
TestContextwith parent context support and revamp cleanup viaRecover/RecoverAndExit - Refactor Banff suite and Antithesis entrypoints to use the new abstractions
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_context.go | Added GetDefaultContextParent() to TestContext interface |
| tests/simple_test_context.go | Added parent context, error/panic handlers, replaced Cleanup with Recover/RecoverAndExit |
| tests/load/c/main/main.go | Switched defer from tc.Cleanup() to tc.RecoverAndExit() |
| tests/fixture/e2e/ginkgo_test_context.go | Registered GinkgoTestContext and implemented GetDefaultContextParent() |
| tests/fixture/e2e/apitest.go | Defined APITestFunction and ExecuteAPITest helper |
| tests/e2e/banff/suites.go | Refactored Banff test into TestCustomAssetTransfer and invoked via ExecuteAPITest |
| tests/context_helpers.go | Updated ContextWithTimeout to respect TestContext parent context |
| tests/antithesis/xsvm/main.go | Switched to antithesis context and RecoverAndExit(), set parent context |
| tests/antithesis/context.go | Added NewInstrumentedTestContextWithArgs with Antithesis assertion handlers |
| tests/antithesis/avalanchego/main.go | Adopted Antithesis context abstraction and replaced wallet init with e2e.NewWallet |
Comments suppressed due to low confidence (1)
tests/simple_test_context.go:70
- [nitpick] Consider adding unit tests for
RecoverAndExitandRecoverto verify cleanup invocation and correct panic/exit behavior under assertion failures.
func (tc *SimpleTestContext) RecoverAndExit() {
|
Approving my own PR is a treat (because my personal account proposed it), but I'll hold off on merging until someone else approves. |
| // execTestWithRecovery ensures assertion-related panics encountered during test execution are recovered | ||
| // and that deferred cleanups are always executed before returning. | ||
| func execTestWithRecovery(ctx context.Context, log logging.Logger, test Test, wallet *Wallet) { | ||
| tc := tests.NewTestContext(log) |
There was a problem hiding this comment.
Creating a new TestContext here ensures that recovery executes only the cleanups registered to this context.
| @@ -85,10 +85,7 @@ func (l LoadGenerator) Run( | |||
| default: | |||
| } | |||
|
|
|||
| ctx, cancel := context.WithTimeout(ctx, testTimeout) | |||
There was a problem hiding this comment.
Moved to execTestWithRecovery to ensure that cancelation is localized to test execution rather than only being performed on goroutine exit.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: maru <maru.newby@avalabs.org>
Why this should be merged
Having to duplicate existing test coverage to benefit from execution with antithesis is not ideal. Better to be able to trivially refactor compatible e2e coverage for reuse. This PR refactors the banff e2e test for reuse with antithesis to demonstrate that this is possible.
How this works
APITestFunctiontype that a compatible e2e test can implement to allow for execution with both ginkgo and antithesisExecuteAPITesthelper to simplify execution of anAPITestFunctionwith ginkgoAPITestFunctionexecuted byExecuteAPITestHow this was tested
CI
Need to be documented in RELEASES.md?
N/A