Validate RHEL 9.2

[caroline-suse-rancher]

K3s Versions to be Validated

• 1.27
• 1.26

Testing Considerations

QA to change as necessary

1. Install and run sonobuoy conformance tests on a hardened cluster
2. Validate SUC upgrade
3. Install Rancher Manager

Additional Information

Jira Ticket: https://jira.suse.com/browse/SURE-6485

This should be validated with selinux enabled:

# /etc/rancher/k3s/config.yaml
selinux: true

[aganesh-suse]

RHEL 9.2.
Configs:

protect-kernel-defaults: true
selinux: true

Edited kernel params:

sudo bash -c 'cat <<EOF> /etc/sysctl.d/90-kubelet.conf
vm.panic_on_oom=0
vm.overcommit_memory=1
kernel.panic=10
kernel.panic_on_oops=1
kernel.keys.root_maxbytes=25000000
EOF'
 
sudo sysctl -p /etc/sysctl.d/90-kubelet.conf

Install k3s
Run sonobuoy tests - Results PASS

[aganesh-suse]

SUC Upgrades were successful:

1. Edit kernel parameters as mentioned in previous comment

2. Initial Install setup: v1.25.10+k3s

sudo mkdir -p /etc/rancher/k3s
sudo bash -c 'cat <<EOF> /etc/rancher/k3s/config.yaml
write-kubeconfig-mode: "0644"
debug: true
token: secret
cluster-init: true
protect-kernel-defaults: true
selinux: true
node-label:
  - k3s-upgrade=server
EOF'


Servers:

sudo mkdir -p /etc/rancher/k3s
sudo bash -c 'cat <<EOF> /etc/rancher/k3s/config.yaml
write-kubeconfig-mode: "0644"
debug: true
token: secret
protect-kernel-defaults: true
selinux: true
server: https://ec2-52-14-14-10.us-east-2.compute.amazonaws.com:6443/
node-label:
  - k3s-upgrade=server
EOF'

Agents:

sudo mkdir -p /etc/rancher/k3s
sudo bash -c 'cat <<EOF> /etc/rancher/k3s/config.yaml
debug: true
token: secret
protect-kernel-defaults: true
selinux: true
server: https://ec2-52-14-14-10.us-east-2.compute.amazonaws.com:6443/
node-label:
  - k3s-upgrade=agent
EOF'

curl -fL https://get.k3s.io/| INSTALL_K3S_VERSION=$version sh -s -

3. Used system controller:

kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/download/[v0.11.0](https://github.com/rancher/system-upgrade-controller/releases/tag/v0.11.0-dev.1)/system-upgrade-controller.yaml

4. Created workloads.
5. Applied a plan.yaml to upgrade the servers and workers
   from v1.25.10+k3s -> v1.26.5+k3s -> v1.27.2+k3s versions successfully.
   Plan.yaml:

apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: k3s-server
  namespace: system-upgrade
  labels:
    k3s-upgrade: server
spec:
  concurrency: 1
  version: v1.27.2+k3s1
  nodeSelector:
    matchExpressions:
      - {key: node-role.kubernetes.io/master, operator: In, values: ["true"]}
  serviceAccountName: system-upgrade
  cordon: true
  #drain:
  #  force: true
  upgrade:
    image: rancher/k3s-upgrade
---
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: k3s-agent
  namespace: system-upgrade
  labels:
    k3s-upgrade: agent
spec:
  concurrency: 1
  version: v1.27.2+k3s1
  nodeSelector:
    matchExpressions:
      - {key: node-role.kubernetes.io/master, operator: NotIn, values: ["true"]}
  serviceAccountName: system-upgrade
  prepare:
    image: rancher/k3s-upgrade
    args: ["prepare", "k3s-server"]
  upgrade:
    image: rancher/k3s-upgrade

[aganesh-suse]

3 server/ 1 agent setup

Installed k3s v1.26.5+k3s1

with sample server config.yaml:

sudo mkdir -p /etc/rancher/k3s
sudo bash -c 'cat <<EOF> /etc/rancher/k3s/config.yaml
write-kubeconfig-mode: "0644"
debug: true
token: secret
cluster-init: true
selinux: true
node-label:
  - k3s-upgrade=server
tls-san: "18.117.117.224.nip.io"
node-external-ip: 18.117.117.224
EOF'

Then helm installed rancher:

helm repo add rancher-latest https://releases.rancher.com/server-charts/latest && \
helm repo add jetstack https://charts.jetstack.io && \
helm repo update && \
kubectl create namespace cattle-system && \
kubectl create namespace cert-manager && \
kubectl apply --validate=false -f https://github.com/jetstack/cert-manager/releases/download/v1.12.1/cert-manager.crds.yaml && \
helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v1.12.1
helm install rancher rancher-latest/rancher \
  --namespace cattle-system \
  --set hostname=18.117.117.224.nip.io \
  --set rancherImageTag=v2.7-head \
  --version=v2.7.5-rc2

Was able to get to rancher UI