Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validate Rocky 9.2 #7734

Closed
caroline-suse-rancher opened this issue Jun 10, 2023 · 1 comment
Closed

Validate Rocky 9.2 #7734

caroline-suse-rancher opened this issue Jun 10, 2023 · 1 comment
Assignees
@endawkins
Labels
kind/os-validation
Milestone
v1.27.4+k3s1

Comments

@caroline-suse-rancher
Copy link
Contributor

caroline-suse-rancher commented Jun 10, 2023 •
edited by rancher-max
Loading

K3s Versions to be Validated

  • 1.27
  • 1.26

Testing Considerations

QA to change as necessary

  1. Install and run sonobuoy conformance tests
  2. Validate SUC upgrade

Additional Information

Jira Ticket: https://jira.suse.com/browse/SURE-6485

This should be validated with selinux enabled:

# /etc/rancher/k3s/config.yaml
selinux: true
@caroline-suse-rancher caroline-suse-rancher added this to the v1.27.4+k3s1 milestone Jun 10, 2023
@endawkins
Copy link

Validated on branch <branch_name> with commit <commit_id> / version

Environment Details

Infrastructure

  • Cloud
  • Hosted

Node(s) CPU architecture, OS, and Version:

Linux ip-172-31-21-173.us-east-2.compute.internal 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 17:09:15 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Rocky Linux"
VERSION="9.2 (Blue Onyx)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.2"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Rocky Linux 9.2 (Blue Onyx)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:9::baseos"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2032-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-9"
ROCKY_SUPPORT_PRODUCT_VERSION="9.2"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.2"

Cluster Configuration:

3 servers 1 agent (HA Configuration)

Config.yaml:

server 1:
cluster-init: true
write-kubeconfig-mode: 644
selinux: true

server 2 & 3:
write-kubeconfig-mode: 644
server: "https://<serverIP>:6443"
token: test
selinux: true

agent:
server: "https://<serverIP>:6443"
token: test
selinux: true

Additional files

v1.26.5+k3s1 plan.yaml

apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: k3s-server
  namespace: system-upgrade
  labels:
    k3s-upgrade: server
spec:
  concurrency: 3
  version: v1.26.6-rc1+k3s1
  nodeSelector:
    matchExpressions:
      - {key: node-role.kubernetes.io/master, operator: In, values: ["true"]}
  serviceAccountName: system-upgrade
  cordon: true
  #drain:
  #  force: true
  upgrade:
    image: rancher/k3s-upgrade
---
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: k3s-agent
  namespace: system-upgrade
  labels:
    k3s-upgrade: agent
spec:
  concurrency: 1
  version: v1.26.6-rc1+k3s1
  nodeSelector:
    matchExpressions:
      - {key: node-role.kubernetes.io/master, operator: NotIn, values: ["true"]}
  serviceAccountName: system-upgrade
  prepare:
    image: rancher/k3s-upgrade
    args: ["prepare", "k3s-server"]
  # drain:
    # podSelector:
    #   matchExpressions:
    #     - key: app.kubernetes.io/name
    #       operator: In
    #       values: 
    #         - clusterip-pod-demo
    #         - clusterip-pod-demo-2
      # matchLabels:
      # app: clusterip-demo
      #  app.kubernetes.io/kind: pod
      # label can be only 1 key/value pair
    # force: true
    # ignoreDaemonSets: false
    # skipWaitForDeleteTimeout: 60
    # deleteEmptyDir: true
    # deleteLocalDir: true
  upgrade:
    image: rancher/k3s-upgrade

v1.27.2+k3s1 plan.yaml:

apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: k3s-server
  namespace: system-upgrade
  labels:
    k3s-upgrade: server
spec:
  concurrency: 3
  version: v1.27.3-rc1+k3s1
  nodeSelector:
    matchExpressions:
      - {key: node-role.kubernetes.io/master, operator: In, values: ["true"]}
  serviceAccountName: system-upgrade
  cordon: true
  #drain:
  #  force: true
  upgrade:
    image: rancher/k3s-upgrade
---
apiVersion: upgrade.cattle.io/v1
kind: Plan
metadata:
  name: k3s-agent
  namespace: system-upgrade
  labels:
    k3s-upgrade: agent
spec:
  concurrency: 1
  version: v1.27.3-rc1+k3s1
  nodeSelector:
    matchExpressions:
      - {key: node-role.kubernetes.io/master, operator: NotIn, values: ["true"]}
  serviceAccountName: system-upgrade
  prepare:
    image: rancher/k3s-upgrade
    args: ["prepare", "k3s-server"]
  # drain:
    # podSelector:
    #   matchExpressions:
    #     - key: app.kubernetes.io/name
    #       operator: In
    #       values: 
    #         - clusterip-pod-demo
    #         - clusterip-pod-demo-2
      # matchLabels:
      # app: clusterip-demo
      #  app.kubernetes.io/kind: pod
      # label can be only 1 key/value pair
    # force: true
    # ignoreDaemonSets: false
    # skipWaitForDeleteTimeout: 60
    # deleteEmptyDir: true
    # deleteLocalDir: true
  upgrade:
    image: rancher/k3s-upgrade

Testing Steps

  1. Copy config.yaml
$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s
  1. Install k3s
  2. Run sonobuoy test
sudo yum install wget (if wget is not already installed)
wget https://github.com/vmware-tanzu/sonobuoy/releases/download/v0.56.16/sonobuoy_0.56.16_linux_amd64.tar.gz
tar -xvf sonobuoy_0.56.16_linux_amd64.tar.gz
sudo chmod +x sonobuoy && sudo mv sonobuoy /usr/local/bin/sonobuoy
sonobuoy run --kubernetes-version=<VERSION>--mode=certified-conformance
sonobuoy status --kubeconfig <path to config>
  1. Run SUC upgrade
kubectl apply -f https://github.com/rancher/system-upgrade-controller/releases/download/v0.11.0-dev.3/system-upgrade-controller.yaml

kubectl label node -l node-role.kubernetes.io/master==true k3s-upgrade=server && kubectl label node -l node-role.kubernetes.io/master!=true k3s-upgrade=agent

kubectl apply -f plan.yaml

Replication Results:

  • k3s version used for replication:
N/A

N/A

Validation Results:

  • k3s version used for validation:
v1.26.5 (Sonobuoy)
v1.27.2 (Sonobuoy)
v1.26.5+k3s1 -> v1.26.6-rc1+k3s1 (SUC)
v1.27.2+k3s1 -> v1.27.3-rc1+k3s1 (SUC)

***Sonobuoy Results:
v1.26.5+k3s1:

 PLUGIN     STATUS   RESULT   COUNT                 PROGRESS
            e2e   complete   passed       1   Passed:368, Failed:  0
   systemd-logs   complete   passed       4

v1.27.2+k3s1:

PLUGIN     STATUS   RESULT   COUNT                                PROGRESS
            e2e   complete   passed       1   Passed:378, Failed:  0, Remaining:  0
   systemd-logs   complete   passed       4
   
***SUC Upgrade Results:
v1.26.5+k3s1 -> v1.26.6-rc1+k3s1
NAME                                               STATUS   ROLES                       AGE     VERSION            INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                      KERNEL-VERSION                 CONTAINER-RUNTIME
node/ip-172-31-18-67.us-east-2.compute.internal    Ready    control-plane,etcd,master   3h26m   v1.26.6-rc1+k3s1   172.31.18.67    <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1
node/ip-172-31-21-173.us-east-2.compute.internal   Ready    control-plane,etcd,master   3h34m   v1.26.6-rc1+k3s1   172.31.21.173   <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1
node/ip-172-31-21-20.us-east-2.compute.internal    Ready    control-plane,etcd,master   3h24m   v1.26.6-rc1+k3s1   172.31.21.20    <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1
node/ip-172-31-26-181.us-east-2.compute.internal   Ready    <none>                      3h28m   v1.26.6-rc1+k3s1   172.31.26.181   <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1


v1.27.2+k3s1 -> v1.27.3-rc1+k3s1
NAME                                               STATUS   ROLES                       AGE     VERSION            INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                      KERNEL-VERSION                 CONTAINER-RUNTIME
node/ip-172-31-23-219.us-east-2.compute.internal   Ready    control-plane,etcd,master   5m57s   v1.27.3-rc1+k3s1   172.31.23.219   <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1
node/ip-172-31-23-251.us-east-2.compute.internal   Ready    control-plane,etcd,master   5m42s   v1.27.3-rc1+k3s1   172.31.23.251   <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1
node/ip-172-31-27-160.us-east-2.compute.internal   Ready    control-plane,etcd,master   7m44s   v1.27.3-rc1+k3s1   172.31.27.160   <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1
node/ip-172-31-29-252.us-east-2.compute.internal   Ready    <none>                      6m9s    v1.27.3-rc1+k3s1   172.31.29.252   <none>        Rocky Linux 9.2 (Blue Onyx)   5.14.0-284.1                   1.1.el9_2.x86_64   containerd://1.7.1-k3s1

Additional context / logs:

N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@endawkins endawkins

Labels
kind/os-validation
Projects
K3s Development
Archived in project
Milestone
v1.27.4+k3s1
Development

No branches or pull requests
2 participants
@caroline-suse-rancher @endawkins