Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: add installation, cluster setup, emojivoto demo; fix mdx component mapping, prism theme #266

Merged
merged 6 commits into from
Mar 26, 2024
Merged

docs: add installation, cluster setup, emojivoto demo; fix mdx component mapping, prism theme #266

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
629ca0b
docs: add cluster setup to getting started category
katexochen Mar 20, 2024
fe8fe63
docs: update intro
katexochen Mar 20, 2024
c3f0886
docs: add emojivoto example
katexochen Mar 20, 2024
099a4c9
docs/theme: fix MDXComponent mapping for docusaurus v3
katexochen Mar 20, 2024
6bb672b
docs/config: fix prism code theme
katexochen Mar 20, 2024
6a9703b
docs: add installation instructions
katexochen Mar 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ Contrast is based on the [Kata Containers](https://github.com/kata-containers/ka
[Confidential Containers](https://github.com/confidential-containers) projects.
Confidential Containers are Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation from the surrounding environment.
This works with unmodified containers in a lift-and-shift approach.
It currently targets the [CoCo preview on AKS](https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-containers-on-aks-preview).
Contrast currently targets the [CoCo preview on AKS](https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-containers-on-aks-preview).

## Goal

Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer, i.e., remove the infrastructure from the TCB. This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, e.g., malicious co-tenants escalating their privileges.
Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer, i.e., remove the infrastructure from the trusted computing base (TCB). This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, e.g., malicious co-tenants escalating their privileges.

Contrast integrates fluently with the existing Kubernetes workflows. It's compatible with managed Kubernetes, can be installed as a day-2 operation and imposes only minimal changes to your deployment flow.

Expand Down
170 changes: 0 additions & 170 deletions dev-docs/user-manual.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,3 @@
# Contrast

Contrast runs confidential container deployments on untrusted Kubernetes at scale.

Contrast is based on the [Kata Containers](https://github.com/kata-containers/kata-containers) and
[Confidential Containers](https://github.com/confidential-containers) projects. Confidential Containers are
Kubernetes pods that are executed inside a confidential micro-VM and provide strong hardware-based isolation
from the surrounding environment. This works with unmodified containers in a lift-and-shift approach.
It currently targets the [CoCo preview on AKS](https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-containers-on-aks-preview).

## Goal

Contrast is designed to keep all data always encrypted and to prevent access from the infrastructure layer, i.e., remove the infrastructure from the TCB. This includes access from datacenter employees, privileged cloud admins, own cluster administrators, and attackers coming through the infrastructure, e.g., malicious co-tenants escalating their privileges.

Contrast integrates fluently with the existing Kubernetes workflows. It's compatible with managed Kubernetes, can be installed as a day-2 operation and imposes only minimal changes to your deployment flow.

## Use Cases:

* Increasing the security of your containers
* Moving sensitive workloads from on-prem to the cloud with Confidential Computing
* Shielding the code and data even from the own cluster administrators
* Increasing the trustworthiness of your SaaS offerings
* Simplifying regulatory compliance
* Multi-party computation for data collaboration

## Features

### 🔒 Everything always encrypted
Expand Down Expand Up @@ -102,151 +77,6 @@ After the last step, we know that the policy has not been tampered with and, thu
Contrast provides an Initializer that handles the remote attestation on the workload side transparently and
fetches the workload certificate. The Initializer runs as an init container before your workload is started.

## How to deploy emojivoto in Contrast

### Prerequisites

Install the latest version of the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/).
[Login to your account](https://docs.microsoft.com/en-us/cli/azure/authenticate-azure-cli), which has the permissions to create an AKS cluster, by
executing:

```sh
az login
```

Create an AKS cluster with Confidential Container support:

```sh
# Ensure you set this to an existing resource group in your subscription
azResourceGroup="ContrastDemo"
# Select the name for your AKS cluster
azClusterName="ContrastDemo"

az extension add \
--name aks-preview \
--allow-preview true

az extension update \
--name aks-preview \
--allow-preview true

az feature register --namespace "Microsoft.ContainerService" --name "KataCcIsolationPreview"
az feature show --namespace "Microsoft.ContainerService" --name "KataCcIsolationPreview"
az provider register -n Microsoft.ContainerService

az aks create \
--resource-group "$azResourceGroup" \
--name "$azClusterName" \
--kubernetes-version 1.29 \
--os-sku AzureLinux \
--node-vm-size Standard_DC4as_cc_v5 \
--node-count 1 \
--generate-ssh-keys

az aks nodepool add \
--resource-group "$azResourceGroup" \
--name nodepool2 \
--cluster-name "$azClusterName" \
--mode System \
--node-count 1 \
--os-sku AzureLinux \
--node-vm-size Standard_DC4as_cc_v5 \
--workload-runtime KataCcIsolation

az aks get-credentials \
--resource-group "$azResourceGroup" \
--name "$azClusterName"
```

### Download the latest Contrast release

Since Contrast has not yet been publicly released, a download URL will be
provided by Edgeless.

```sh
wget <URL>
unzip contrast.zip
```

### Deploy the Contrast Coordinator

Install the latest Contrast Coordinator release, comprising a single replica
deployment and a LoadBalancer service, into your cluster:

```sh
kubectl apply -f coordinator.yml
```

### Annotate the emojivoto deployment

Run the generate command to generate the execution policies and add them as
annotations to your deployment files. A manifest.json with the reference values
of your deployment will be created:

```sh
./contrast generate deployment/*
```

### Set the manifest

Attest the Coordinator and set the manifest. It might take up to a few minutes
for the load balancer to be created.

```sh
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $coordinator
./contrast set -c "${coordinator}:1313" -m manifest.json deployment/
```

### Deploy emojivoto

Since the coordinator has a manifest set, which defines the emojivoto deployment as an allowed workload, we can deploy the application:

```sh
kubectl apply -f deployment/
```

### Verify the Coordinator

An end user (data owner) can verify the Contrast deployment using the verify
command:

```sh
./contrast verify -c "${coordinator}:1313"
```

The CLI will attest the Coordinator using embedded reference values. The CLI
will write the Coordinator's CA root certificate (`mesh-root.pem`) and the history of manifests into
the `verify/` directory. In addition, the policies referenced in the manifest are
also written into the same directory.

### Connect and verify the workload

You can securely connect to the workloads using the Coordinator's `mesh-root.pem` as a trusted CA certificate.
First, expose the service on a public IP address via a LoadBalancer service:

```sh
kubectl patch svc web-svc -p '{"spec": {"type": "LoadBalancer"}}'
timeout 30s bash -c 'until kubectl get service/web-svc --output=jsonpath='{.status.loadBalancer}' | grep "ingress"; do sleep 2 ; done'
lbip=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $lbip
```

Note: All workload certificates are created with a wildcard DNS entry. Since we are accessing the load balancer via IP, the SAN checks the certificate for IP entries in the SAN field. Since the certificate doesn't contain any IP entries as SAN, the validation fails.
Hence, with curl you need to skip the validation:

```sh
curl -k "https://${lbip}:443"
```

To validate the certificate with the `mesh-root.pem` locally, use `openssl` instead:

```sh
openssl s_client -showcerts -connect ${lbip}:443 </dev/null | sed -n -e '/-.BEGIN/,/-.END/ p' > certChain.pem
awk 'BEGIN {c=0;} /BEGIN CERT/{c++} { print > "cert." c ".pem"}' < certChain.pem
openssl verify -verbose -trusted verify/mesh-root.pem -- cert.1.pem
```

## Current limitations

Contrast is in an early preview stage, and most underlying projects are still under development as well.
Expand Down
161 changes: 161 additions & 0 deletions docs/docs/examples/emojivoto.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# Confidential voting

<!-- TODO(katexochen): create a screenshot with fixed emoji font and fixed format -->
![screenshot of the emojivoto UI](../../static/img/emoijvoto.png)

This tutorial guides you through deploying [emojivoto](https://github.com/BuoyantIO/emojivoto) as a
confidential Contrast deployment. Emojivoto is an example app that allows users to vote for different emojis and view votes cast on a leaderboard. It has a microservice architecture consisting of a
web frontend (`web`), a gRPC backend for listing available emojis (`emoji`), and a backend for
the voting and leaderboard logic (`voting`). The `vote-bot` simulates user traffic by submitting
votes to the frontend.

<!-- TODO(katexochen): recreate in our design -->
![emojivoto components topology](https://github.com/BuoyantIO/emojivoto/e490d5789086e75933a474b22f9723fbfa0b29ba/assets/emojivoto-topology.png)

### Motivation

Using a voting service, users' votes are considered highly sensitive data, as we require
a secret ballot. Also, users are likely interested in the fairness of the ballot. For
both requirements, we can use Confidential Computing and, specifically, workload attestation
to prove to those interested in voting that the app is running in a protected environment
where their votes are processed without leaking to the platform provider or workload owner.

## Prerequisites

- **Installed Contrast CLI.**
See the [installation instructions](./../getting-started/install.md) on how to get it.
- **Running cluster with Confidential Containers support.**
Please follow the [cluster setup instructions](./../getting-started/cluster-setup.md)
for creating a cluster.
- **Get the deployment.** This is currently available as part of the preview bundle.

## Steps to deploy emojivoto with Contrast

### Deploy the Contrast Coordinator

Deploy the Contrast Coordinator, comprising a single replica deployment and a
LoadBalancer service, into your cluster:

```sh
kubectl apply -f coordinator.yml
```

### Generate policy annotations and manifest

Run the `generate` command to generate the execution policies and add them as
annotations to your deployment files. A `manifest.json` file with the reference values
of your deployment will be created:

```sh
contrast generate deployment/
```

:::note[Runtime class and Initializer]

The deployment YAML shipped for this demo is already configured to be used with Contrast.
A runtime class `kata-cc-isolation` was added to the pods to signal they should be run
as Confidential Containers. In addition, the Contrast Initializer was added
as an init container to these workloads to facilitate the attestation and certificate pulling
before the actual workload is started.

:::

### Set the manifest

Configure the coordinator with a manifest. It might take up to a few minutes
for the load balancer to be created and the Coordinator being available.

```sh
coordinator=$(kubectl get svc coordinator -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo "The user API of your Contrast Coordinator is available at $coordinator:1313"
contrast set -c "${coordinator}:1313" deployment/
```

The CLI will use the embedded reference values to attest the Coordinator deployment
during the TLS handshake. If the connection succeeds, we are ensured that the Coordinator
deployment hasn't been tampered with.

### Deploy emojivoto

Now that the coordinator has a manifest set, which defines the emojivoto deployment as an allowed workload,
we can deploy the application:

```sh
kubectl apply -f deployment/
```

:::note[Inter-deployment communication]

The Contrast Coordinator issues mesh certificates after successfully validating workloads.
These certificates can be used for sercure inter-deployment communication. The Initializer
sends an attestation report to the Coordinator, retrieves certificates and a private key in return
and writes them to a volumeMount. The emojivoto version we are using is patched to only communicate
via mTLS (the original app talks plain HTTP). The different parts of the workload are configured
to use the credentials from the volumeMount when communicating with each other.

:::

## Voter's perspective: verifying the ballot

As voters, we want to verify the fairness and confidentiality of the deployment before
deciding to vote. Regardless of the scale of our distributed deployment, Contrast only
needs a single remote attestation step to verify the deployment. By doing remote attestation
of the Coordinator, we transitively verify those systems the Coordinator has already attested
or will attest in the future. Successful verification of the Coordinator means that
we can be sure it will enforce the configured manifest.

### Attest the Coordinator

A potential voter can verify the Contrast deployment using the verify
command:

```sh
contrast verify -c "${coordinator}:1313"
```

The CLI will attest the Coordinator using embedded reference values. If the command succeeds,
the Coordinator deployment was successfully verified to be running in the expected Confidential
Computing environment with the expected code version. The Coordinator will then return its
configuration over the established TLS channel. The CLI will store this information, namely the root
certificate of the mesh (`mesh-root.pem`) and the history of manifests, into the `verify/` directory.
In addition, the policies referenced in the manifest history are also written into the same directory.

### Manifest history and artifact audit

In the next step, the Coordinator configuration that was written by the `verify` command needs to be audited.
A potential voter should inspect the manifest and the referenced policies. They could delegate
this task to an entity they trust.

### Confidential connection to the attested workload

After ensuring the configuration of the Coordinator fits the expectation, you can securely connect
to the workloads using the Coordinator's `mesh-root.pem` as a trusted CA certificate.

To access the web frontend, expose the service on a public IP address via a LoadBalancer service:

```sh
kubectl patch svc web-svc -p '{"spec": {"type": "LoadBalancer"}}'
timeout 30s bash -c 'until kubectl get service/web-svc --output=jsonpath='{.status.loadBalancer}' | grep "ingress"; do sleep 2 ; done'
lbip=$(kubectl get svc web-svc -o=jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo $lbip
```

:::info

By default, certificates are issued with a wildcard DNS entry. Since we are accessing the load balancer via IP, the SAN checks the certificate for IP entries in the SAN field. Since the certificate doesn't contain any IP entries as SAN, the validation fails.
Hence, certificate validation will fail with using curl:

```sh
$ curl --cacert ./verify/mesh-root.pem "https://${lbip}:443"
curl: (60) SSL: no alternative certificate subject name matches target host name '203.0.113.34'
```

:::

To validate the certificate with the `mesh-root.pem` locally, use `openssl` instead:

```sh
openssl s_client -showcerts -connect ${lbip}:443 </dev/null | sed -n -e '/-.BEGIN/,/-.END/ p' > certChain.pem
awk 'BEGIN {c=0;} /BEGIN CERT/{c++} { print > "cert." c ".pem"}' < certChain.pem
openssl verify -verbose -trusted verify/mesh-root.pem -- cert.1.pem
Comment on lines +158 to +160
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this have an advantage over just calling openssl s_client -CAfile verify/mesh-root.pem -verify_return_error -connect ${lbip}:443 </dev/null?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@3u13r

```
Loading