Support Server Token Rotation

[dereknola]

K3s Tracking for rancher/rke2#2141

[ShylajaDevadiga]

OS: Ubuntu 22.04
Cluster Config: Single node
Commit: dface01

Following the steps for non-root user,
new-token is updated to the token file.
update token in service file

$ curl -fL https://get.k3s.io/ | INSTALL_K3S_COMMIT=dface01de8390ca294c14926452576ddcda6b959 sh -s - server --cluster-init --token token1

$ k3s token --help
NAME:
   k3s token - Manage bootstrap tokens

USAGE:
   k3s token command [command options] [arguments...]

COMMANDS:
   create    Create bootstrap tokens on the server
   delete    Delete bootstrap tokens on the server
   generate  Generate and print a bootstrap token, but do not create it on the server
   list      List bootstrap tokens on the server
   rotate    Rotate original server token with a new bootstrap token

OPTIONS:
   --help, -h  show help

$ k3s token rotate --token token1 --new-token=token2
WARNING: Recommended to keep a record of the old token. If restoring from a snapshot, you must use the token associated with that snapshot.
WARN[0000] Cluster CA certificate is not trusted by the host CA bundle, but the token does not include a CA hash. Use the full token from the server's node-token file to enable Cluster CA validation. 
Token rotated, restart k3s with new token

$ sudo cat /var/lib/rancher/k3s/server/token
K107b0d86f8cc6cf595a35b8225ae91be501feb613e0a3febdbfa1a3423fd74788c::server:token2

Updated k3s service file with new token before restarting k3s

$ sudo systemctl restart k3s

$ kubectl get nodes
NAME              STATUS   ROLES                       AGE   VERSION
ip-172-31-0-209   Ready    control-plane,etcd,master   13m   v1.28.2+k3s-dface01d

[ShylajaDevadiga]

Validated using commit id dface01

• On single server as non-root and root user
• On HA with 3 server 1 agent, passing token as well as on server generated token
• Validated cluster came up clean after reboot