Skip to content

Latest commit

 

History

History
76 lines (64 loc) · 6.88 KB

kubernetes-eks.md

File metadata and controls

76 lines (64 loc) · 6.88 KB

Deploying Malcolm on Amazon Elastic Kubernetes Service (EKS)

This document outlines the process of setting up a cluster on Amazon Elastic Kubernetes Service (EKS) using Amazon Web Services in preparation for Deploying Malcolm with Kubernetes.

This is a work-in-progress document that is still a bit rough around the edges. Users will need to replace things such as cluster-name and us-east-1 with the values that are appliable to the cluster. Any feedback is welcome in the relevant issue on GitHub.

This document assumes good working knowledge of Amazon Web Services (AWS) and Amazon Elastic Kubernetes Service (EKS). Good documentation resources can be found in the AWS documentation, the EKS documentation and the EKS Workshop.

Prerequisites

  • aws cli - the AWS Command Line Interface with functioning access to the AWS infrastructure
  • eksctl - the official CLI for Amazon EKS

Procedure

  1. Create a Virtual Private Cloud (VPC)

    • subnets in at least 2 availability zones
    • tag private subnets with kubernetes.io/role/internal-elb: 1
    • tag public subnets with kubernetes.io/role/elb: 1
    • enable "auto-assign public IP address" for public subnets
  2. Create a security group for the VPC

  3. Create an Elastic Kubernetes Service (EKS) cluster

  4. Generate a kubeconfig file to use with Malcolm's control scripts (malcolmeks.yaml is used in this example)

    aws eks update-kubeconfig --region us-east-1 --name cluster-name --kubeconfig malcolmeks.yaml
  5. Create a node group

    • Both c4.4xlarge and t3a.2xlarge seem to be good instance types for Malcolm, but users' needs may vary (see recommended system requirements for Malcolm)
    • set the nodes to run on the VPC's public subnets
  6. Deploy metrics-server

    kubectl --kubeconfig=malcolmeks.yaml apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
  7. Associate IAM OIDC provider with cluster

    eksctl utils associate-iam-oidc-provider --region=us-east-1 --cluster=cluster-name --approve
  8. Deploy the AWS Load Ballancer Controller add-on

  9. deploy Amazon EFS CSI driver

    • review Prerequisites
    • follow steps for Create an IAM policy and role
    • follow steps for Install the Amazon EFS driver
    • follow steps for Create an Amazon EFS file system
  10. Set up access points, and note the Access point IDs to put in the YAML in the next step

    name mountpoint access point ID
    config /malcolm/config fsap-…
    opensearch /malcolm/opensearch fsap-…
    opensearch-backup /malcolm/opensearch-backup fsap-…
    pcap /malcolm/pcap fsap-…
    runtime-logs /malcolm/runtime-logs fsap-…
    suricata-logs /malcolm/suricata-logs fsap-…
    zeek-logs /malcolm/zeek-logs fsap-…
  11. Create manifest for persistent volumes and volume claims from the EFS file system ID and access point IDs

    • See PersistentVolumeClaim Definitions under Deploying Malcolm with Kubernetes
    • [kubernetes/01-volumes-aws-efs.yml.example]({{ site.github.repository_url }}/blob/{{ site.github.build_revision }}/kubernetes/01-volumes-aws-efs.yml.example) is an example manifest to use as a starting point. Copy 01-volumes-aws-efs.yml.example to 01-volumes.yml and replace fs-FILESYSTEMID with the EFS file system and each fsap-… value with the corresponding access point ID from the previous step.
  12. Finish the configuration then start Malcolm as described in Deploying Malcolm with Kubernetes

Attribution

Amazon Web Services, AWS, the Powered by AWS logo, and Amazon Elastic Kubernetes Service (EKS) are trademarks of Amazon.com, Inc. or its affiliates. The information about providers and services contained in this document is for instructional purposes and does not constitute endorsement or recommendation.