[Release-1.25] - Helm controller bootstrap option is broken when using port other than 6443

[dereknola]

Backport fix for Helm controller bootstrap option is broken when using port other than 6443

• Helm controller bootstrap option is broken when using port other than 6443 #7835

[est-suse]

Validated on branch 1.25 c3eab73

Environment Details

Infrastructure
Cloud EC2 instance


Node(s) CPU architecture, OS, and Version:
Ubuntu 22.04

Cluster Configuration:
Multi node: 3 server 1 agent

Steps:

Install k3s passing --https-listen-port to port other than default port 6443
Deploy a chart with bootstrap option

apiVersion: v1
kind: Namespace
metadata:
  name: web
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: apache
  namespace: kube-system
spec:
  bootstrap: true
  repo: https://charts.bitnami.com/bitnami
  chart: apache
  targetNamespace: web
  valuesContent: |-
    service:
      type: ClusterIP
    ingress:
      enabled: true
      hostname: www.example.com
    metrics:
      enabled: true

Verify the cluster is up and chart is deployed correctly
Results from reproducing using k3s v1.25.11+k3s1:

curl -fL https://get.k3s.io| INSTALL_K3S_VERSION=v1.25.11+k3s1 sh -s - server --cluster-init --https-listen-port 7443

$ cat /etc/rancher/k3s/k3s.yaml |grep 744
    server: https://127.0.0.1:7443

$ kubectl get nodes
NAME               STATUS   ROLES                       AGE   VERSION
ip-1-1-1-1   Ready    control-plane,etcd,master   39m   v1.25.11+k3s1

$ kubectl get pods -A
NAMESPACE     NAME                                     READY   STATUS             RESTARTS       AGE
kube-system   coredns-77ccd57875-p7cln                 1/1     Running            0              39m
kube-system   helm-install-apache-ptwqk                0/1     CrashLoopBackOff   11 (32s ago)   32m
kube-system   helm-install-traefik-74mj8               0/1     Completed          1              39m
kube-system   helm-install-traefik-crd-djtxl           0/1     Completed          0              39m
kube-system   local-path-provisioner-957fdf8bc-rklft   1/1     Running            0              39m
kube-system   metrics-server-648b5df564-4n7rn          1/1     Running            0              39m
kube-system   svclb-traefik-17c427a4-xx4jm             2/2     Running            0              39m
kube-system   traefik-64f55bb67d-7q6ls                 1/1     Running            0              39m


if [[ ${KUBERNETES_SERVICE_HOST} =~ .*:.* ]]; then
	echo "KUBERNETES_SERVICE_HOST is using IPv6"
	CHART="${CHART//%\{KUBERNETES_API\}%/[${KUBERNETES_SERVICE_HOST}]:${KUBERNETES_SERVICE_PORT}}"
else
	CHART="${CHART//%\{KUBERNETES_API\}%/${KUBERNETES_SERVICE_HOST}:${KUBERNETES_SERVICE_PORT}}"
fi

set +v -x
+ [[ true != \t\r\u\e ]]
+ [[ '' == \1 ]]
+ [[ '' == \v\2 ]]
+ shopt -s nullglob
+ [[ -f /config/ca-file.pem ]]
+ [[ -f /tmp/ca-file.pem ]]
+ [[ -n '' ]]
+ helm_content_decode
+ set -e
+ ENC_CHART_PATH=/chart/apache.tgz.base64
+ CHART_PATH=/tmp/apache.tgz
+ [[ ! -f /chart/apache.tgz.base64 ]]
+ return
+ [[ install != \d\e\l\e\t\e ]]
+ helm_repo_init
+ grep -q -e 'https\?://'
+ [[ helm_v3 == \h\e\l\m\_\v\3 ]]
+ [[ apache/apache == stable/* ]]
+ [[ -n https://charts.bitnami.com/bitnami ]]
+ [[ -f /auth/username ]]
+ helm_v3 repo add apache https://charts.bitnami.com/bitnami
"apache" has been added to your repositories
+ helm_v3 repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "apache" chart repository
Update Complete. ⎈Happy Helming!⎈
+ helm_update install --namespace web
+ [[ helm_v3 == \h\e\l\m\_\v\3 ]]
++ helm_v3 ls --all -f '^apache$' --namespace ++ web --output json
tr '[:upper:]' '[:lower:]'
++ jq -r '"\(.[0].app_version),\(.[0].status)"'
Error: Kubernetes cluster unreachable: Get "https://127.0.0.1:6443/version": dial tcp 127.0.0.1:6443: connect: connection refused
+ LINE=
+ IFS=,
+ read -r INSTALLED_VERSION STATUS _
+ VALUES=
+ for VALUES_FILE in /config/*.yaml
+ VALUES=' --values /config/values-01_HelmChart.yaml'
+ [[ install = \d\e\l\e\t\e ]]
+ [[ '' =~ ^(|null)$ ]]
+ [[ '' =~ ^(|null)$ ]]
+ echo 'Installing helm_v3 chart'
+ helm_v3 install --namespace web apache apache/apache --values /config/values-01_HelmChart.yaml
Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "https://127.0.0.1:6443/version": dial tcp 127.0.0.1:6443: connect: connection refused

Results from validation using latest commit:

NAME                    STATUS   ROLES                       AGE    VERSION
node/ip-172-31-18-9     Ready    control-plane,etcd,master   159m   v1.25.11+k3s-c3eab737
node/ip-172-31-19-33    Ready    control-plane,etcd,master   171m   v1.25.11+k3s-c3eab737
node/ip-172-31-26-93    Ready    control-plane,etcd,master   170m   v1.25.11+k3s-c3eab737
node/ip-172-31-28-108   Ready    <none>                      165m   v1.25.11+k3s-c3eab737

NAMESPACE     NAME                                          READY   STATUS      RESTARTS   AGE
kube-system   pod/coredns-8b9777675-jp2zl                   1/1     Running     0          170m
kube-system   pod/helm-install-apache-psrdz               0/1     Completed   0          157m
kube-system   pod/helm-install-traefik-crd-wgsl7            0/1     Completed   0          170m
kube-system   pod/helm-install-traefik-dfmgm                0/1     Completed   1          170m
kube-system   pod/local-path-provisioner-69dff9496c-98kbs   1/1     Running     0          170m
kube-system   pod/metrics-server-854c559bd-p78dq            1/1     Running     0          170m
kube-system   pod/svclb-traefik-5193b956-5xbp8              2/2     Running     0          170m
kube-system   pod/svclb-traefik-5193b956-gn562              2/2     Running     0          170m
kube-system   pod/svclb-traefik-5193b956-ms5lb              2/2     Running     0          158m
kube-system   pod/svclb-traefik-5193b956-qhcnt              2/2     Running     0          165m
kube-system   pod/traefik-66fd46ccd-bg8m9                   1/1     Running     0          170m
web           pod/apache-6c5c56c8f8-8mnhd                   2/2     Running     0          157m

cat /etc/rancher/k3s/k3s.yaml |grep 744
    server: https://127.0.0.1:7443
kubectl describe pod -n kube-system helm-install-apache-psrdz|grep  PORT
      KUBERNETES_SERVICE_PORT:  7444