[tmpnet] Enable externally accessible URIs for kube-hosted nodes

[maru-ava]

Why this should be merged

Previously, access from outside of a kube cluster was enabled by port-forwarding through the kube API. This approach turned out incompatible with load testing because it greatly limited the rate at which transactions could be sent.

The port-forwarding is replaced with nginx+ingress to ensure minimum overhead when running outside of a kube cluster.

How this works

• deploy kind cluster with a static NodePort
• deploy the nginx ingress operator with helm and configure to use the NodePort
• update kube runtime to create a service and ingress for each node to configure external accessibility via nginx
• replace GetLocalURI with simplified GetAccessibleURI
• enabled configuration of the base accessible URI to enable use of a hosted kube cluster

How this was tested

CI

Need to be documented in RELEASES.md?

N/A

TODO

• Perform self-review
• Ensure attribution for kind-with-registry.sh
• Merge [tmpnet] Source tmpnet defaults from a configmap #4057

[Copilot]

Pull Request Overview

This PR replaces port-forwarding with an externally accessible URI approach for kube-hosted nodes in order to improve load testing performance. Key changes include refactoring functions from GetLocalURI to GetAccessibleURI (and staking address equivalents), adding new logic to create Kubernetes Service and Ingress resources, and updating tests to use the new accessible URI methods.

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/fixture/tmpnet/utils.go	Updated error handling in CheckNodeHealth and simplified Node URI retrieval
tests/fixture/tmpnet/start_kind_cluster.go	Added functions to deploy and verify the nginx ingress controller via Helm
tests/fixture/tmpnet/process_runtime.go	Replaced GetLocalURI with GetAccessibleURI and updated staking address method
tests/fixture/tmpnet/node.go	Updated Node interface methods to use the accessible URI versions
tests/fixture/tmpnet/network.go	Updated network URI retrieval to use GetAccessibleURI
tests/fixture/tmpnet/kube_runtime.go	Refactored URI retrieval for nodes to support external access and added Service/Ingress
tests/fixture/tmpnet/flags/kube_runtime.go	Introduced new flag for base accessible URI configuration
tests/e2e/*	Updated tests to use GetAccessibleURI (and staking address) throughout
scripts/kind-with-registry.sh	Added a new script to set up kind with a local registry
flake.nix	Removed kind-with-registry derivation and updated PATH configuration

[RodrigoVillar]

Ran this locally with the load test passing.

I tried using this against the remote cluster with no success (this is expected). However, I believe you mentioned that this PR should refuse deployment to a context other than kind-kind. Do you think we should add the refuse logic to this PR?

[maru-ava]

Ran this locally with the load test passing.

I tried using this against the remote cluster with no success (this is expected). However, I believe you mentioned that this PR should refuse deployment to a context other than kind-kind. Do you think we should add the refuse logic to this PR?

I meant to say that the test should only target kind-kind, not that tmpnet should refuse to deploy to a context other than kind-kind.

Was the failure something like failed to wait for Ingress...? It may make sense to log a warning if the ingress status suggests that the reason is the lack of an ingress controller.

[RodrigoVillar]

@maru-ava I'm getting the following error:

[06-26|09:29:52.291] ERROR avalanchego-load-test tests/simple_test_context.go:33
        Error Trace:    /Users/rodrigo.villar/go/src/github.com/ava-labs/avalanchego/tests/fixture/e2e/helpers.go:304
                                                /Users/rodrigo.villar/go/src/github.com/ava-labs/avalanchego/tests/fixture/e2e/env.go:185
                                                /Users/rodrigo.villar/go/src/github.com/ava-labs/avalanchego/tests/load/c/main/main.go:63
                                                /Users/rodrigo.villar/.goenv/versions/1.23.9/src/runtime/proc.go:272
                                                /Users/rodrigo.villar/.goenv/versions/1.23.9/src/runtime/asm_arm64.s:1223
        Error:          Received unexpected error:
                        failed to query node health: failed to issue request: Post "http://localhost:30791/networks/cb4ff442-bea0-401b-b5b5-5d7c38fc1e9c/NodeID-PWaVaDZqkNCfsYBtdCsXBSBkdas6Uj3Fu/ext/health": dial tcp 127.0.0.1:30791: connect: connection refused
        Messages:       failed to bootstrap network

Going back to the original question, logging a warning would be great here.

[maru-ava]

Rebased

[maru-ava]

Going back to the original question, logging a warning would be great here.

I added more checks, but when I was testing this, I realized that our test cluster actually has an ingress controller so the checks didn't actually help.

Instead, I've changed how the base accessible URI is configured so that it is no longer defaulted to the kind-kind value. Now if you try to deploy outside a kube cluster and the target cluster is not the kind cluster we deploy, it will complain that --base-accessible-uri must be provided and exit without deploying anything.

Edit: Updated to source the ingress configuration from a configmap in the cluster. That way it can be zero-config for the end-user.

[maru-ava]

Dropped basicauth protection - the guid and node id should be sufficient.