Fix connection-draining flakes in TestAllocatorAfterDeleteReplica / /TestGameServerCreationRightAfterDeletingOneExtensionsPod #3853
Labels
kind/bug
These are bugs.
Comments
zmerlynn
added a commit
to zmerlynn/agones
that referenced
this issue
Jun 5, 2024
…ealth check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853
zmerlynn
added a commit
to zmerlynn/agones
that referenced
this issue
Jun 5, 2024
…ealth check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853
zmerlynn
added a commit
to zmerlynn/agones
that referenced
this issue
Jun 11, 2024
…ealth check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853
zmerlynn
added a commit
to zmerlynn/agones
that referenced
this issue
Jun 26, 2024
…ealth check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853
markmandel
pushed a commit
that referenced
this issue
Jul 12, 2024
* agones-{extensions,allocator}: Make servers context aware, add gRPC health check
* adds an `httpserver` utility package to handle the `Run` function
that controller/extensions use. Make that context aware using the same
method as https.Run:
https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130
* also plumbs context-awareness through the allocator
run{Mux,REST,GRPC} functions.
* adds a gRPC health server to the allocator, calls .Shutdown() on it
during graceful termination - this seems to push the client off correctly.
Tested with e2e in a loop.
Towards #3853
* Move from context.Background()
* Use Shutdown/GracefulStop
* Relax deadline slightly (original had none), also delete pod from GetAllocatorClient
indexjoseph
pushed a commit
to indexjoseph/agones
that referenced
this issue
Jul 12, 2024
…ames#3845) * agones-{extensions,allocator}: Make servers context aware, add gRPC health check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853 * Move from context.Background() * Use Shutdown/GracefulStop * Relax deadline slightly (original had none), also delete pod from GetAllocatorClient
indexjoseph
pushed a commit
to indexjoseph/agones
that referenced
this issue
Jul 12, 2024
… apis (googleforgames#3900) * Adds a command to generate the zz_generated.deepcopy.go files for the apis * Combines CRD client and deepcopy code generation into one script * Updates boilerplate headers to 2024 * Generated code from gen-all-sdk-grpc * Small update to template comment agones-{extensions,allocator}: Make servers context aware (googleforgames#3845) * agones-{extensions,allocator}: Make servers context aware, add gRPC health check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853 * Move from context.Background() * Use Shutdown/GracefulStop * Relax deadline slightly (original had none), also delete pod from GetAllocatorClient Update Slack invite link (googleforgames#3896) Looks like the Slack invite link expired somewhere along the way, so this is a new, shiny updated one!
indexjoseph
pushed a commit
to indexjoseph/agones
that referenced
this issue
Jul 12, 2024
… apis (googleforgames#3900) * Adds a command to generate the zz_generated.deepcopy.go files for the apis * Combines CRD client and deepcopy code generation into one script * Updates boilerplate headers to 2024 * Generated code from gen-all-sdk-grpc * Small update to template comment agones-{extensions,allocator}: Make servers context aware (googleforgames#3845) * agones-{extensions,allocator}: Make servers context aware, add gRPC health check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853 * Move from context.Background() * Use Shutdown/GracefulStop * Relax deadline slightly (original had none), also delete pod from GetAllocatorClient Update Slack invite link (googleforgames#3896) Looks like the Slack invite link expired somewhere along the way, so this is a new, shiny updated one!
indexjoseph
pushed a commit
to indexjoseph/agones
that referenced
this issue
Jul 12, 2024
… apis (googleforgames#3900) * Adds a command to generate the zz_generated.deepcopy.go files for the apis * Combines CRD client and deepcopy code generation into one script * Updates boilerplate headers to 2024 * Generated code from gen-all-sdk-grpc * Small update to template comment agones-{extensions,allocator}: Make servers context aware (googleforgames#3845) * agones-{extensions,allocator}: Make servers context aware, add gRPC health check * adds an `httpserver` utility package to handle the `Run` function that controller/extensions use. Make that context aware using the same method as https.Run: https://github.com/googleforgames/agones/blob/dfa414e5e4da37798833bbf8c33919acb5f3c2ea/pkg/util/https/server.go#L127-L130 * also plumbs context-awareness through the allocator run{Mux,REST,GRPC} functions. * adds a gRPC health server to the allocator, calls .Shutdown() on it during graceful termination - this seems to push the client off correctly. Tested with e2e in a loop. Towards googleforgames#3853 * Move from context.Background() * Use Shutdown/GracefulStop * Relax deadline slightly (original had none), also delete pod from GetAllocatorClient Update Slack invite link (googleforgames#3896) Looks like the Slack invite link expired somewhere along the way, so this is a new, shiny updated one!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I've been fighting a 2-3% flake in these tests, both related to our servers not seemingly to terminate connections during graceful shutdown. Common errors include:
rpc error: code = Unavailable desc = error reading from server: EOFand
rpc error: code = Unavailable desc = error reading from server: read tcp 192.168.25.113:36748->104.198.58.226:443: read: connection reset by peer#3839 and #3843 were both towards fixing this but do not seem to have succeeded.
The text was updated successfully, but these errors were encountered: