fix: use correct wasm shims names

[flavio]

Proposed Changes

Change how containerd configuration is generated with regards to WebAssembly shims.

Types of Changes

Use correct binary names and the associated RuntimeType. Prior to this, there was quite a mixup between v1 and v2 shims.

Verification

• Download spin and the slight shims from here
• Unpack the tarball and move the files to /usr/bin/containerd-shim-spin-v2 and /usr/bin/containerd-shim-slight-v1
• Install/restart the patched version of k3s

Check the contents of /var/lib/rancher/k3s/agent/etc/containerd/config.toml, it should have the following lines:

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes."slight"]
  runtime_type = "io.containerd.slight.v1"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes."slight".options]
  BinaryName = "/usr/bin/containerd-shim-slight-v1"
  SystemdCgroup = true

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes."spin"]
  runtime_type = "io.containerd.spin.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes."spin".options]
  BinaryName = "/usr/bin/containerd-shim-spin-v2"
  SystemdCgroup = true

Then, to test the slight-shim is actually working, create the following Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: wasm-slight
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wasm-slight
  template:
    metadata:
      labels:
        app: wasm-slight
    spec:
      runtimeClassName: slight
      containers:
        - name: slight-hello
          image: ghcr.io/deislabs/containerd-wasm-shims/examples/slight-rust-hello:v0.9.1
          command: ["/"]
          resources: # limit the resources to 128Mi of memory and 100m of CPU
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi

Then, to test the spin-shim is actually working, create the following Deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: wasm-spin
spec:
  replicas: 1
  selector:
    matchLabels:
      app: wasm-spin
  template:
    metadata:
      labels:
        app: wasm-spin
    spec:
      runtimeClassName: spin
      containers:
        - name: spin-hello
          image: ghcr.io/deislabs/containerd-wasm-shims/examples/spin-rust-hello:v0.11.1
          command: ["/"]
          resources: # limit the resources to 128Mi of memory and 100m of CPU
            limits:
              cpu: 100m
              memory: 128Mi
            requests:
              cpu: 100m
              memory: 128Mi

You should now have 2 Pod running.

Both Pods are small web application. To access them we have to create the following resources:

# create a traefik middleware
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: strip-prefix
  namespace: default
spec:
  stripPrefix:
    forceSlash: false
    prefixes:
      - /spin
      - /slight
---
# define slight service
apiVersion: v1
kind: Service
metadata:
  name: wasm-slight
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000
  selector:
    app: wasm-slight
---
# define spin service
apiVersion: v1
kind: Service
metadata:
  name: wasm-spin
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: wasm-spin
---
# define a single ingress, that exposes both services
# using a path route
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wasm-ingress
  annotations:
    ingress.kubernetes.io/ssl-redirect: "false"
    kubernetes.io/ingress.class: traefik
    traefik.ingress.kubernetes.io/router.middlewares: default-strip-prefix@kubernetescrd
spec:
  rules:
    - http:
        paths:
          - path: /slight
            pathType: Prefix
            backend:
              service:
                name: wasm-slight
                port:
                  number: 80
          - path: /spin
            pathType: Prefix
            backend:
              service:
                name: wasm-spin
                port:
                  number: 80

Then run the following commands:

# optain the public IP of the ingress
export PUBLIC_IP=$(kubectl get ingress wasm-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
# verify the slight application
curl -v http://$PUBLIC_IP/slight/hello
# verify the spin application
curl -v http://$PUBLIC_IP/spin/go-hello
curl -v http://$PUBLIC_IP/spin/hello

Testing

This was already covered by the unit tests, however that didn't help with the detection of the issue.

Does k3s have the concept of integration tests? The idea would be to automate the verification steps. I would be happy to write this integration test.

Linked Issues

• K3s uses incorrect wasm shims names #9650

User-Facing Change

Further Comments

CC @brandond and @vitorsavian who initially reviewed #8751

[brandond]

Does k3s have the concept of integration tests? The idea would be to automate the verification steps. I would be happy to write this integration test.

You might take a look at https://github.com/k3s-io/k3s/tree/master/tests/integration and https://github.com/k3s-io/k3s/tree/master/tests/e2e and see where you think this might fit best.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 42.92%. Comparing base (085ccbb) to head (33e541c).
Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9519      +/-   ##
==========================================
- Coverage   49.12%   42.92%   -6.20%     
==========================================
  Files         151      154       +3     
  Lines       13475    13523      +48     
==========================================
- Hits         6619     5805     -814     
- Misses       5515     6550    +1035     
+ Partials     1341     1168     -173     

Flag	Coverage Δ
e2etests	?
inttests	39.49% <100.00%> (+<0.01%)	⬆️
unittests	15.96% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dereknola]

Since this can be validated on a single server node, this should be written as an integration test.

[flavio]

Since this can be validated on a single server node, this should be written as an integration test.

The test needs to download some binaries from the internet and put them on specific locations on the filesystem. From what I've seen, the integration tests are not doing that (see for example the longhorn one, which bails out if iscsiadm is not installed).

The e2e tests, on the other hand, seem to allow this kind of operations.

Because of that, I was more inclined towards writing a e2e tests.

Thoughts?

[dereknola]

Hmm, thats fair, we don't want to add binaries to a host without them knowing, so E2E it is.

[flavio]

@brandond , @dereknola: I've added a new e2e test. I've taken inspiration from other tests that are already inside of the code base. Let me know if there's anything I have to change to make it more idiomatic

[dereknola]

You accidentally butchered the LICENSE and README files. Can you revert those changes?

[flavio]

@dereknola doh! sorry about that...

I've fixed the issue with a force push

[flavio]

It seems the amount of e2e tests ran by drone is limited (see here).

Is that done by purpose?

Should I add my new e2e test over there?

[Mossaka]

I am assuming that this change was not in v1.28.7+k3s1, right?

[flavio]

@Mossaka no, they are not part of a tagged release of k3s. They will be part of the next release