[release-1.26] Ensure that no embedded controllers are using the admin RBAC

[brandond]

• Ensure that no embedded controllers are using the admin RBAC #7212 backport

[endawkins]

Validated on branch 1.26 with commit 3b1919b / version 1.26

Environment Details

Infrastructure

• Cloud
• Hosted

Node(s) CPU architecture, OS, and Version:

Linux ip-172-31-21-173.us-east-2.compute.internal 5.14.0-284.11.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Tue May 9 17:09:15 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Rocky Linux"
VERSION="9.2 (Blue Onyx)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.2"
PLATFORM_ID="platform:el9"
PRETTY_NAME="Rocky Linux 9.2 (Blue Onyx)"
ANSI_COLOR="0;32"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:rocky:rocky:9::baseos"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
SUPPORT_END="2032-05-31"
ROCKY_SUPPORT_PRODUCT="Rocky-Linux-9"
ROCKY_SUPPORT_PRODUCT_VERSION="9.2"
REDHAT_SUPPORT_PRODUCT="Rocky Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.2"

Cluster Configuration:

1 server, 1 agent (optional)

Config.yaml:

cluster-init: true
write-kubeconfig-mode: 644
selinux: true

Additional files

sudo cat /var/lib/rancher/k3s/server/manifests/web-helmchart.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: web
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: apache
  namespace: kube-system
spec:
  repo: https://charts.bitnami.com/bitnami
  chart: apache
  targetNamespace: web
  valuesContent: |-
    service:
      type: ClusterIP
    ingress:
      enabled: true
      hostname: www.example.com
    metrics:
      enabled: true

Testing Steps

1. Copy config.yaml

$ sudo mkdir -p /etc/rancher/k3s && sudo cp config.yaml /etc/rancher/k3s

2. Install k3s
3. Run the following commands:

sudo ls /var/lib/rancher/k3s/server/cred
sudo ls /var/lib/rancher/k3s/server/tls
sudo cat /var/lib/rancher/k3s/server/manifests/web-helmchart.yaml
kubectl get pods -n web
sudo mkdir -p -m 700 /var/lib/rancher/k3s/server/logs
sudo cat /var/lib/rancher/k3s/server/audit.yaml
sudo cat /var/lib/rancher/k3s/server/logs/audit.log | grep supervisor

Replication Results:

• k3s version used for replication:

k3s -v
k3s version v1.26.5+k3s1 (7cefebea)
go version go1.19.9

sudo ls /var/lib/rancher/k3s/server/cred
admin.kubeconfig  api-server.kubeconfig  cloud-controller.kubeconfig  controller.kubeconfig  ipsec.psk	passwd	scheduler.kubeconfig

sudo ls /var/lib/rancher/k3s/server/tls
client-admin.crt       client-ca.nochain.crt		client-k3s-controller.key  client-scheduler.crt   server-ca.crt		      serving-kube-apiserver.key
client-admin.key       client-controller.crt		client-kube-apiserver.crt  client-scheduler.key   server-ca.key		      serving-kubelet.key
client-auth-proxy.crt  client-controller.key		client-kube-apiserver.key  dynamic-cert.json	  server-ca.nochain.crt       temporary-certs
client-auth-proxy.key  client-k3s-cloud-controller.crt	client-kube-proxy.crt	   etcd			  service.current.key
client-ca.crt	       client-k3s-cloud-controller.key	client-kube-proxy.key	   request-header-ca.crt  service.key
client-ca.key	       client-k3s-controller.crt	client-kubelet.key	   request-header-ca.key  serving-kube-apiserver.crt

sudo cat /var/lib/rancher/k3s/server/manifests/web-helmchart.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: web
---
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: apache
  namespace: kube-system
spec:
  repo: https://charts.bitnami.com/bitnami
  chart: apache
  targetNamespace: web
  valuesContent: |-
    service:
      type: ClusterIP
    ingress:
      enabled: true
      hostname: www.example.com
    metrics:
      enabled: true


kubectl get pods -n web
NAME                     READY   STATUS    RESTARTS   AGE
apache-bc5c7bb89-f2hk6   2/2     Running   0          16m

Validation Results:

• k3s version used for validation:

k3s -v
k3s version v1.26.6-rc1+k3s1 (3b1919b0)
go version go1.19.10

sudo mkdir -p -m 700 /var/lib/rancher/k3s/server/logs

sudo ls /var/lib/rancher/k3s/server/cred
admin.kubeconfig  api-server.kubeconfig  cloud-controller.kubeconfig  controller.kubeconfig  ipsec.psk	passwd	scheduler.kubeconfig  supervisor.kubeconfig

sudo cat /var/lib/rancher/k3s/server/audit.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata

sudo ls /var/lib/rancher/k3s/server/tls
client-admin.crt       client-ca.key			client-k3s-cloud-controller.key  client-kube-proxy.crt	client-supervisor.crt  request-header-ca.key  service.key
client-admin.key       client-ca.nochain.crt		client-k3s-controller.crt	 client-kube-proxy.key	client-supervisor.key  server-ca.crt	      serving-kube-apiserver.crt
client-auth-proxy.crt  client-controller.crt		client-k3s-controller.key	 client-kubelet.key	dynamic-cert.json      server-ca.key	      serving-kube-apiserver.key
client-auth-proxy.key  client-controller.key		client-kube-apiserver.crt	 client-scheduler.crt	etcd		       server-ca.nochain.crt  serving-kubelet.key
client-ca.crt	       client-k3s-cloud-controller.crt	client-kube-apiserver.key	 client-scheduler.key	request-header-ca.crt  service.current.key    temporary-certs


sudo cat /var/lib/rancher/k3s/server/logs/audit.log | grep supervisor

{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Metadata","auditID":"3290dbf8-311f-4fd2-80fb-85e206035164","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/kube-system/events","verb":"create","user":{"username":"system:k3s-supervisor","groups":["system:masters","system:authenticated"]},"sourceIPs":["127.0.0.1"],"userAgent":"deploy@ip-172-31-21-173.us-east-2.compute.internal/v1.26.6-rc1+k3s1 (linux/amd64) k3s/3b1919b0","objectRef":{"resource":"events","namespace":"kube-system","name":"web-helmchart.176b6b4c3ffc662d","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":201},"requestReceivedTimestamp":"2023-06-23T22:47:43.190726Z","stageTimestamp":"2023-06-23T22:47:43.193700Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":""}}

Additional context / logs:

N/A