Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

--set controller.watchIngressWithoutClass=true missing from rke2 nginx-ingress latest chart preventing ingress from obtaining externalIPs #6214

Closed
VestigeJ opened this issue Jun 18, 2024 · 2 comments
Closed

--set controller.watchIngressWithoutClass=true missing from rke2 nginx-ingress latest chart preventing ingress from obtaining externalIPs #6214

VestigeJ opened this issue Jun 18, 2024 · 2 comments
Assignees
@VestigeJ
Labels
kind/bug-qa status/release-blocker
Milestone
v1.30.2+rke2r1

Comments

@VestigeJ
Copy link
Contributor

Environmental Info:
RKE2 Version:

all latest RCs

Node(s) CPU architecture, OS, and Version:

N/A

Cluster Configuration:

basic rke2 deployment 3 control planes 1 agent -

Describe the bug:

The rke2-ingress-nginx.yaml chart is missing the " --set controller.watchIngressWithoutClass=true " prevents the deployed ingresses from correctly grabbing the external NodeIPs.

Steps To Reproduce:

  • Installed RKE2: using any of the latest RCs

deploy rancher v2.8.3,4, etc observe the ingress doesn't obtain the external IPs after deployment applying the controller.watchIngressWithoutClass=true arg back to the chart to fix.

Expected behavior:

the ingress for the deployment correctly receives the nodeIPs to expose services on.

Actual behavior:

The ingress doesn't contain any externalIPs to LoadBalance traffic on.

Additional context / logs:

$ kg ing rancher -n cattle-system -o yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/issuer: rancher
    cert-manager.io/issuer-kind: Issuer
    meta.helm.sh/release-name: rancher
    meta.helm.sh/release-namespace: cattle-system
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
  creationTimestamp: "2024-06-18T15:50:43Z"
  generation: 1
  labels:
    app: rancher
    app.kubernetes.io/managed-by: Helm
    chart: rancher-2.8.4
    heritage: Helm
    release: rancher
  name: rancher
  namespace: cattle-system
  resourceVersion: "4723"
  uid: 2ed1f8f1-07b7-4f1e-8f99-de0a9df481c0
spec:
  rules:
  - host: matracerack3.qa
    http:
      paths:
      - backend:
          service:
            name: rancher
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - matracerack3.qa
    secretName: tls-rancher-ingress
status:
  loadBalancer: {}

$ kg ing rancher -n cattle-system -o yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/issuer: rancher
    cert-manager.io/issuer-kind: Issuer
    field.cattle.io/publicEndpoints: '[{"addresses":["3.6.1.1","3.6.9.9","3.6.8.8","3.6.5.5"],"port":443,"protocol":"HTTPS","serviceName":"cattle-system:rancher","ingressName":"cattle-system:rancher","hostname":"ip-3-6-1-1","path":"/","allNodes":false}]'
    meta.helm.sh/release-name: rancher
    meta.helm.sh/release-namespace: cattle-system
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
  creationTimestamp: "2024-06-18T17:36:55Z"
  generation: 1
  labels:
    app: rancher
    app.kubernetes.io/managed-by: Helm
    chart: rancher-2.8.4
    heritage: Helm
    release: rancher
  name: rancher
  namespace: cattle-system
  resourceVersion: "9346"
  uid: fc31001f-e42b-4ff8-aa1b-eda76290d464
spec:
  rules:
  - host: ip-3-6-1-1
    http:
      paths:
      - backend:
          service:
            name: rancher
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - ip-3-6-1-1
    secretName: tls-rancher-ingress
status:
  loadBalancer:
    ingress:
    - ip: 3.6.1.1
    - ip: 3.6.9.9
    - ip: 3.6.8.8
    - ip: 3.6.5.5
@brandond
Copy link
Member

brandond commented Jun 18, 2024 •
edited
Loading

The change in rancher/rke2-charts#463 to switch controller.watchIngressWithoutClass from true to false was not intended to land in advance of #5943, but @dereknola bumped the ingress-nginx chart to 4.10.1 on top of that change, which rolled it out sooner than intended. I should have coordinated that better with the team.

We should default that to true again, until next month when we merge #5943 in the July cycle.

This was referenced Jun 18, 2024
Merged
Merged
Merged
Merged
Merged
@brandond brandond added this to the v1.30.2+rke2r1 milestone Jun 18, 2024
@VestigeJ VestigeJ self-assigned this Jun 20, 2024
@VestigeJ
Copy link
Contributor Author

VestigeJ commented Jun 21, 2024 •
edited
Loading

This has been resolved on the latest RCs

NAME                     STATUS   ROLES                       AGE   VERSION
node/ip-2-2-18-255   Ready    control-plane,etcd,master   15m   v1.28.11+rke2r1
node/ip-2-2-22-255   Ready    control-plane,etcd,master   11m   v1.28.11+rke2r1
node/ip-2-2-5-53     Ready    <none>                      12m   v1.28.11+rke2r1
node/ip-2-2-8-157    Ready    control-plane,etcd,master   11m   v1.28.11+rke2r1

NAMESPACE      NAME                                                       READY   STATUS      RESTARTS   AGE
cert-manager   pod/cert-manager-cainjector-698464d9bb-h9bhh               1/1     Running     0          3m54s
cert-manager   pod/cert-manager-d7db49bf4-jczfp                           1/1     Running     0          3m54s
cert-manager   pod/cert-manager-webhook-f6c9958d-888gw                    1/1     Running     0          3m54s
kube-system    pod/cloud-controller-manager-ip-2-2-18-255                 1/1     Running     0          15m
kube-system    pod/cloud-controller-manager-ip-2-2-22-255                 1/1     Running     0          11m
kube-system    pod/cloud-controller-manager-ip-2-2-8-157                  1/1     Running     0          10m
kube-system    pod/etcd-ip-2-2-18-255                                     1/1     Running     0          14m
kube-system    pod/etcd-ip-2-2-22-255                                     1/1     Running     0          10m
kube-system    pod/etcd-ip-2-2-8-157                                      1/1     Running     0          10m
kube-system    pod/helm-install-rke2-canal-6qj2w                          0/1     Completed   0          15m
kube-system    pod/helm-install-rke2-coredns-glmkd                        0/1     Completed   0          15m
kube-system    pod/helm-install-rke2-ingress-nginx-7wfmq                  0/1     Completed   0          15m
kube-system    pod/helm-install-rke2-metrics-server-8skrc                 0/1     Completed   0          15m
kube-system    pod/helm-install-rke2-snapshot-controller-8fp6s            0/1     Completed   1          15m
kube-system    pod/helm-install-rke2-snapshot-controller-crd-zgc8z        0/1     Completed   0          15m
kube-system    pod/helm-install-rke2-snapshot-validation-webhook-vwbvw    0/1     Completed   0          15m
kube-system    pod/kube-apiserver-ip-2-2-18-255                           1/1     Running     0          15m
kube-system    pod/kube-apiserver-ip-2-2-22-255                           1/1     Running     0          11m
kube-system    pod/kube-apiserver-ip-2-2-8-157                            1/1     Running     0          11m
kube-system    pod/kube-controller-manager-ip-2-2-18-255                  1/1     Running     0          15m
kube-system    pod/kube-controller-manager-ip-2-2-22-255                  1/1     Running     0          11m
kube-system    pod/kube-controller-manager-ip-2-2-8-157                   1/1     Running     0          10m
kube-system    pod/kube-proxy-ip-2-2-18-255                               1/1     Running     0          15m
kube-system    pod/kube-proxy-ip-2-2-22-255                               1/1     Running     0          11m
kube-system    pod/kube-proxy-ip-2-2-5-53                                 1/1     Running     0          12m
kube-system    pod/kube-proxy-ip-2-2-8-157                                1/1     Running     0          11m
kube-system    pod/kube-scheduler-ip-2-2-18-255                           1/1     Running     0          15m
kube-system    pod/kube-scheduler-ip-2-2-22-255                           1/1     Running     0          11m
kube-system    pod/kube-scheduler-ip-2-2-8-157                            1/1     Running     0          10m
kube-system    pod/rke2-canal-bpq6m                                       2/2     Running     0          12m
kube-system    pod/rke2-canal-db24z                                       2/2     Running     0          14m
kube-system    pod/rke2-canal-dmxr9                                       2/2     Running     0          11m
kube-system    pod/rke2-canal-m9cjb                                       2/2     Running     0          11m
kube-system    pod/rke2-coredns-rke2-coredns-84b9cb946c-88tbw             1/1     Running     0          14m
kube-system    pod/rke2-coredns-rke2-coredns-84b9cb946c-cz6kq             1/1     Running     0          12m
kube-system    pod/rke2-coredns-rke2-coredns-autoscaler-b49765765-ckzpl   1/1     Running     0          14m
kube-system    pod/rke2-ingress-nginx-controller-7t8r4                    1/1     Running     0          13m
kube-system    pod/rke2-ingress-nginx-controller-llhzc                    1/1     Running     0          10m
kube-system    pod/rke2-ingress-nginx-controller-qbrhj                    1/1     Running     0          10m
kube-system    pod/rke2-ingress-nginx-controller-sc4zn                    1/1     Running     0          11m
kube-system    pod/rke2-metrics-server-655477f655-h8b2g                   1/1     Running     0          14m
kube-system    pod/rke2-snapshot-controller-59cc9cd8f4-rcgh9              1/1     Running     0          13m
kube-system    pod/rke2-snapshot-validation-webhook-54c5989b65-hkmm7      1/1     Running     0          13m

NAMESPACE       NAME                                              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
cattle-system   service/rancher                                   ClusterIP   10.43.37.94    <none>        80/TCP,443/TCP   3m33s
cert-manager    service/cert-manager                              ClusterIP   10.43.2.56     <none>        9402/TCP         3m54s
cert-manager    service/cert-manager-webhook                      ClusterIP   10.43.164.1    <none>        443/TCP          3m54s
default         service/kubernetes                                ClusterIP   10.43.0.1      <none>        443/TCP          15m
kube-system     service/rke2-coredns-rke2-coredns                 ClusterIP   10.43.0.10     <none>        53/UDP,53/TCP    14m
kube-system     service/rke2-ingress-nginx-controller-admission   ClusterIP   10.43.90.125   <none>        443/TCP          13m
kube-system     service/rke2-metrics-server                       ClusterIP   10.43.3.135    <none>        443/TCP          14m
kube-system     service/rke2-snapshot-validation-webhook          ClusterIP   10.43.48.77    <none>        443/TCP          13m

NAMESPACE       NAME                                CLASS    HOSTS               ADDRESS                                                      PORTS     AGE
cattle-system   ingress.networking.k8s.io/rancher   <none>   ip-2-2-22-255     1.1.4.8,1.1.10.13,1.1.6.4,1.1.7.7                              80, 443   3m33s

$ kg ing rancher -n cattle-system -o yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    cert-manager.io/issuer: rancher
    cert-manager.io/issuer-kind: Issuer
    meta.helm.sh/release-name: rancher
    meta.helm.sh/release-namespace: cattle-system
    nginx.ingress.kubernetes.io/proxy-connect-timeout: "30"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
  creationTimestamp: "2024-06-21T18:24:51Z"
  generation: 1
  labels:
    app: rancher
    app.kubernetes.io/managed-by: Helm
    chart: rancher-2.8.5
    heritage: Helm
    release: rancher
  name: rancher
  namespace: cattle-system
  resourceVersion: "4397"
  uid: ea94404e-fc90-4fbd-a493-b94dd7c86af4
spec:
  rules:
  - host: ip-2-2-22-255
    http:
      paths:
      - backend:
          service:
            name: rancher
            port:
              number: 80
        path: /
        pathType: ImplementationSpecific
  tls:
  - hosts:
    - ip-2-2-22-255
    secretName: tls-rancher-ingress
status:
  loadBalancer:
    ingress:
    - ip: 1.1.4.8
    - ip: 1.1.10.13
    - ip: 1.1.6.4
    - ip: 1.1.7.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@VestigeJ VestigeJ

Labels
kind/bug-qa status/release-blocker
Projects
None yet
Milestone
v1.30.2+rke2r1
Development

No branches or pull requests
3 participants
@brandond @VestigeJ @rancher-max