Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update docs to include bare metal SNP #846

Merged
merged 4 commits into from
Sep 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 14 additions & 7 deletions docs/docs/components/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,18 +42,25 @@ The `handler` field in the Kubernetes `RuntimeClass` instructs containerd not to
Instead, containerd invokes a custom plugin called `containerd-shim-contrast-cc-v2`.
This shim is described in more detail in the [upstream source repository](https://github.com/kata-containers/kata-containers/tree/3.4.0/src/runtime) and in the [containerd documentation](https://github.com/containerd/containerd/blob/main/core/runtime/v2/README.md).

### `cloud-hypervisor` virtual machine manager (VMM)
### Virtual machine manager (VMM)

The `containerd` shim uses [`cloud-hypervisor`](https://www.cloudhypervisor.org) to create a confidential virtual machine for every pod.
This requires the `cloud-hypervisor` binary to be installed on every node (responsibility of the [`node-installer`](#node-installer-daemonset)).
The `containerd` shim uses a virtual machine monitor to create a confidential virtual machine for every pod.
On AKS, Contrast uses [`cloud-hypervisor`](https://www.cloudhypervisor.org).
On bare metal, Contrast uses [`QEMU`](https://www.qemu.org/).
The appropriate files are installed on every node by the [`node-installer`](#node-installer-daemonset).

### `Tardev snapshotter`
### Snapshotters

Contrast uses a special [`containerd` snapshotter](https://github.com/containerd/containerd/tree/v1.7.16/docs/snapshotters/README.md) ([`tardev`](https://github.com/kata-containers/tardev-snapshotter)) to provide container images as block devices to the pod-VM. This snapshotter consists of a host component that pulls container images and a guest component (kernel module) used to mount container images.
Contrast uses [`containerd` snapshotters](https://github.com/containerd/containerd/tree/v1.7.16/docs/snapshotters/README.md) to provide container images to the pod-VM.
Each snapshotter consists of a host component that pulls container images and a guest component used to mount/pull container images.

On AKS, Contrast uses the [`tardev`](https://github.com/kata-containers/tardev-snapshotter) snapshotter to provide container images as block devices to the pod-VM.
The `tardev` snapshotter uses [`dm-verity`](https://docs.kernel.org/admin-guide/device-mapper/verity.html) to protect the integrity of container images.
Expected `dm-verity` container image hashes are part of Contrast runtime policies and are enforced by the kata-agent.
This enables workload attestation by specifying the allowed container image as part of the policy. Read [the chapter on policies](policies.md) for more information.

On bare metal, Contrast uses the [`nydus`](https://github.com/containerd/nydus-snapshotter) snapshotter to store metadata about the images. This metadata is communicated to the guest, so that it can pull the images itself.

### Pod-VM image

Every pod-VM starts with the same guest image. It consists of an IGVM file and a root filesystem.
Expand All @@ -77,8 +84,8 @@ The node-level installation is carried out by the Contrast node-installer
After deploying the installer, it performs the following steps on each node:

- Install the Contrast containerd shim (`containerd-shim-contrast-cc-v2`)
- Install `cloud-hypervisor` as the virtual machine manager (VMM)
- Install an IGVM file for pod-VMs of this class
- Install `cloud-hypervisor` or `QEMU` as the virtual machine manager (VMM)
- Install an IGVM file or separate firmware and kernel files for pod-VMs of this class
- Install a read only root filesystem disk image for the pod-VMs of this class
- Reconfigure `containerd` by adding a runtime plugin that corresponds to the `handler` field of the Kubernetes `RuntimeClass`
- Restart `containerd` to make it aware of the new plugin
49 changes: 46 additions & 3 deletions docs/docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,14 @@
The following instructions will guide you through the process of making an existing Kubernetes deployment
confidential and deploying it together with Contrast.

A running CoCo-enabled cluster is required for these steps, see the [setup guide](./getting-started/cluster-setup.md) on how to set it up.
<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
A running CoCo-enabled cluster is required for these steps, see the [setup guide](./getting-started/cluster-setup.md) on how to set up a cluster on AKS.
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
A running CoCo-enabled cluster is required for these steps, see the [setup guide](./getting-started//bare-metal.md) on how to set up a bare metal cluster.
</TabItem>
</Tabs>

## Deploy the Contrast runtime

Expand All @@ -13,18 +20,36 @@ This consists of a `RuntimeClass` resource and a `DaemonSet` that performs insta
This step is only required once for each version of the runtime.
It can be shared between Contrast deployments.

<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/runtime-aks-clh-snp.yml
```
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/runtime.yml
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/runtime-k3s-qemu-snp.yml
```
</TabItem>
</Tabs>

## Deploy the Contrast Coordinator

Install the latest Contrast Coordinator release, comprising a single replica deployment and a
LoadBalancer service, into your cluster.

<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/coordinator-aks-clh-snp.yml
```
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/coordinator.yml
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/coordinator-k3s-qemu-snp.yml
```
</TabItem>
</Tabs>

## Prepare your Kubernetes resources

Expand Down Expand Up @@ -158,9 +183,18 @@ and the Contrast Service Mesh to all workloads that have a specified configurati
After that, it will generate the execution policies and add them as annotations to your deployment files.
A `manifest.json` with the reference values of your deployment will be created.

<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
```sh
contrast generate --reference-values aks-clh-snp resources/
```
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
```sh
contrast generate --reference-values k3s-qemu-snp resources/
```
</TabItem>
</Tabs>

:::warning
Please be aware that runtime policies currently have some blind spots. For example, they can't guarantee the starting order of containers. See the [current limitations](features-limitations.md#runtime-policies) for more details.
Expand All @@ -176,9 +210,18 @@ depending on how you want to customize your deployment.
You can disable the Initializer injection completely by specifying the
`--skip-initializer` flag in the `generate` command.

<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
```sh
contrast generate --reference-values aks-clh-snp --skip-initializer resources/
```
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
```sh
contrast generate --reference-values k3s-qemu-snp --skip-initializer resources/
```
</TabItem>
</Tabs>

</TabItem>

Expand Down
33 changes: 30 additions & 3 deletions docs/docs/examples/emojivoto.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ where their votes are processed without leaking to the platform provider or work
- **Installed Contrast CLI.**
See the [installation instructions](./../getting-started/install.md) on how to get it.
- **Running cluster with Confidential Containers support.**
Please follow the [cluster setup instructions](./../getting-started/cluster-setup.md)
Please follow the [AKS cluster setup instructions](./../getting-started/cluster-setup.md) or [bare metal setup instructions](./../getting-started/bare metal.md)
to create a cluster.

## Steps to deploy emojivoto with Contrast
Expand All @@ -50,28 +50,55 @@ This consists of a `RuntimeClass` resource and a `DaemonSet` that performs insta
This step is only required once for each version of the runtime.
It can be shared between Contrast deployments.

<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/runtime.yml
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/runtime-aks-clh-snp.yml
```
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/runtime-k3s-qemu-snp.yml
```
</TabItem>
</Tabs>

### Deploy the Contrast Coordinator

Deploy the Contrast Coordinator, comprising a single replica deployment and a
LoadBalancer service, into your cluster:

<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/coordinator-aks-clh-snp.yml
```
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
```sh
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/coordinator.yml
kubectl apply -f https://github.com/edgelesssys/contrast/releases/latest/download/coordinator-k3s-qemu-snp.yml
```
</TabItem>
</Tabs>

### Generate policy annotations and manifest

Run the `generate` command to generate the execution policies and add them as
annotations to your deployment files. A `manifest.json` file with the reference values
of your deployment will be created:

<Tabs queryString="platform">
<TabItem value="aks-clh-snp" label="AKS" default>
```sh
contrast generate --reference-values aks-clh-snp deployment/
```
</TabItem>
<TabItem value="k3s-qemu-snp" label="Bare Metal (SNP)">
```sh
contrast generate --reference-values k3s-qemu-snp deployment/
```
</TabItem>
</Tabs>

:::note[Runtime class and Initializer]

Expand Down
2 changes: 1 addition & 1 deletion docs/docs/features-limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This section lists planned features and current limitations of Contrast.
## Availability

- **Platform support**: At present, Contrast is exclusively available on Azure AKS, supported by the [Confidential Container preview for AKS](https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-containers-on-aks-preview). Expansion to other cloud platforms is planned, pending the availability of necessary infrastructure enhancements.
- **Bare-metal support**: Support for running Contrast on bare-metal Kubernetes will be available soon for AMD SEV and Intel TDX.
- **Bare metal support**: Support for running Contrast on bare metal Kubernetes will be available soon for AMD SEV and Intel TDX.

## Kubernetes features

Expand Down
18 changes: 18 additions & 0 deletions docs/docs/getting-started/bare-metal.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Prepare a bare metal instance

## Hardware and firmware setup

1. Update your BIOS to a version that supports AMD SEV-SNP. Updating to the latest available version is recommended as newer versions will likely contain security patches for AMD SEV-SNP.
katexochen marked this conversation as resolved.
Show resolved Hide resolved
2. Enter BIOS setup to enable SMEE, IOMMU, RMP coverage, and SEV-SNP. Set the SEV-ES ASID Space Limit to a non-zero number (higher is better).
3. Download the latest firmware version for your processor from [AMD](https://www.amd.com/de/developer/sev.html), unpack it, and place it in `/lib/firmware/amd`.

Consult AMD's [Using SEV with AMD EPYC Processors user guide](https://www.amd.com/content/dam/amd/en/documents/epyc-technical-docs/tuning-guides/58207-using-sev-with-amd-epyc-processors.pdf) for more information.

## Kernel Setup

1. Install a kernel with version 6.11 or greater. If you're following this guide before 6.11 has been released, use 6.11-rc3. Don't use 6.11-rc4 - 6.11-rc6 as they contain a regression. 6.11-rc7+ might work.

## K3s Setup

1. Follow the [K3s setup instructions](https://docs.k3s.io/) to create a cluster.
2. Install a block storage provider such as [Longhorn](https://docs.k3s.io/storage#setting-up-longhorn) and mark it as the default storage class.
5 changes: 5 additions & 0 deletions docs/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,11 @@ const sidebars = {
label: 'Cluster setup',
id: 'getting-started/cluster-setup',
},
{
type: 'doc',
label: 'Bare metal setup',
id: 'getting-started/bare-metal',
},
]
},
{
Expand Down