Magi Karpenter

Motivation Link to heading

First, if you played the first generation of Pokemon, you know who and what Magikarp is. Magikarp is a water-type Pokemon that only knows one move until it evolves. Real Pokemon trainers see the struggle of leveling up Magikarp, but the reward is worth it once it transforms. You end up with a lovely Gyarados. A comparison of the two is below.

This post is not to nerd out about Pokemon but to talk about a cluster autoscaling tool called karpenter. Cluster autoscaling is a method where you have a set of processes to monitor your existing and requested workloads and scale up and down to meet your demands and adhere to your budget limits. Cluster autoscaling, in general, is similar to how it is to train a Magikarp, where you struggle to tune and set it up in the beginning, but the benefits of getting your Gyarados are worth it in the end. Karpenter comes equipped as a Gyarado, ready to rumble and tumble to help you become the best Pokemon trainer and prepared to catch them all.

Here is pokemon theme song for the memorires for all my pokemon fans out there.

Let me show you a quick example on how easy to get started

Prequisties Link to heading

• AWS account
• AWS Network configuration setup like vpc, subnets, security groups, NACLS, interfaces, etc..
• EKS cluster to deploy Karpenter
• Terraform
• Helm
• Kubectl

If you are familar with terraform I do have terraform modules that are not public that you can use as referce to create some of the prequisties

• AWS network configuration Create a simple VPC with public, private and infra subnets
• EKS Creates a two node eks managed node group cluster

Steps Link to heading

1. Login into your EKS cluster aws eks --region <<EKS_CLUSTER_REGION>> update-kubeconfig --name <<CLUSTER_NAME>>

2. Install karpenter with helm https://karpenter.sh/docs/getting-started/getting-started-with-karpenter/#4-install-karpenter

3. Create a karpenter node pool and ec2nodeclass. Ensure you update role to point to the Node IAM role that your kubernetesn nodes are using. Update tags to ensure contain your eks cluster name

   role: "" # replace with your cluster name
   subnetSelectorTerms:
       - tags:
           karpenter.sh/discovery: "${CLUSTER_NAME}" # replace with your cluster name
   

   apiVersion: karpenter.sh/v1beta1
   kind: NodePool
   metadata:
   name: default
   spec:
   template:
       spec:
       requirements:
           - key: kubernetes.io/arch
           operator: In
           values: ["amd64"]
           - key: kubernetes.io/os
           operator: In
           values: ["linux"]
           - key: karpenter.sh/capacity-type
           operator: In
           values: ["spot"]
           - key: karpenter.k8s.aws/instance-category
           operator: In
           values: ["c", "m", "r"]
           - key: karpenter.k8s.aws/instance-generation
           operator: Gt
           values: ["2"]
       nodeClassRef:
           name: default
   limits:
       cpu: 1000
   disruption:
       consolidationPolicy: WhenUnderutilized
       expireAfter: 720h # 30 * 24h = 720h
   ---
   apiVersion: karpenter.k8s.aws/v1beta1
   kind: EC2NodeClass
   metadata:
   name: default
   spec:
   amiFamily: AL2 # Amazon Linux 2
   role: "KarpenterNodeRole-${CLUSTER_NAME}" # replace with your cluster name
   subnetSelectorTerms:
       - tags:
           karpenter.sh/discovery: "${CLUSTER_NAME}" # replace with your cluster name
   securityGroupSelectorTerms:
       - tags:
           karpenter.sh/discovery: "${CLUSTER_NAME}" # replace with your cluster name
   

4. Create a sample deployment to test out karpenter

   apiVersion: apps/v1
   kind: Deployment
   metadata:
   name: inflate
   spec:
   replicas: 0
   selector:
       matchLabels:
       app: inflate
   template:
       metadata:
       labels:
           app: inflate
       spec:
       terminationGracePeriodSeconds: 0
       containers:
           - name: inflate
           image: public.ecr.aws/eks-distro/kubernetes/pause:3.7
           resources:
               requests:
               cpu: 1
   

5. Scale deployment and watch karpenter logs kubectl scale deployment inflate --replicas 5 && kubectl logs -f -n "${KARPENTER_NAMESPACE}" -l app.kubernetes.io/name=karpenter -c controller

6. Watch new nodes compute as well kubectl get nodes -w

7. Once you have verified, you can scale down kubectl delete deployment inflate && kubectl logs -f -n "${KARPENTER_NAMESPACE}" -l app.kubernetes.io/name=karpenter -c controller

8. Once nodes have scale down, you can delete your cluster. If you created with terraform, terraform destory is all you needed.

Learnings Link to heading

Karpenter is a powerful tool to know if you are in the AWS ecosystem. I see the potential of having lean clusters without sacrificing scaling. Karpenter can request and provision nodes on the fly. I will use it for personal projects where I pay a fixed price for a small size cluster and deploy applications on spot instances. This cluster will help get additional hands-on experience with Kubernetes and experiment with new EKS features.