How Karpenter optimized the management of our EKS infrastructure on AWS
Companies face daily challenges in managing Kubernetes infrastructure, especially to maintain efficiency and reduce costs. Here at Softplan, we discovered a solution that transforms the way we manage our EKS clusters on AWS: Karpenter.
Challenges in instance management
Before talking about Karpenter it is necessary to take a few steps back and explain a little about what it is about auto scaling de nodes. Suppose we have our cluster with some machines (instances) available running our workloads. What happens if there is a peak in usage in our applications and it is necessary to launch more instances (replicas) of our pods? Without a autoscaling we would need to provision a urge, guide you to join our cluster for there yes ours pods be able to be started on this new instance. Remembering that provisioning an instance is not instantaneous, there is a whole bootstrapped of the machine, network settings and many other things before it becomes fully available.
Okay, we've talked about peak user numbers in our applications, but what about when there's idle time? Do we really want to leave these nodes standing with underutilized computing power? To solve this and other issues, the concept of auto scalers.
Auto Scalers
The implementations of auto scalers are basically responsible for the provisioning and consolidation of nodes. Here we are talking about horizontal scaling, that is, adding more machines to our cluster. There are several implementations of node autoscaling, but in this article the focus will be on the implementation of AWS and why we decided to migrate to another solution. Below is a figure exemplifying how node works autoscaling:
Figure 01: autoscaling
AWS – Auto Scaling Groups
When defining a scaling group in the AWS We need to define several properties, such as the minimum/maximum number of node instances allowed for this group, resources used, disk type, network configurations (subnets, etc.) and many other details. For example, for a certain type of application that uses more CPU, we will configure a group that contains instance types with more CPU than memory. In the end, we will probably have several distinct groups for certain types of applications.
Putting the pieces together – Cluster Auto Scaler
So that my cluster be able to “talk” to my cloud providers (in this example AWS), we need a component called Cluster Auto Scaler, or CAS.This component was created by the community that maintains the Kubernetes, and is available here.
A default configuration of the CAS can be seen below, using the helmet for installation:
nameOverride: cluster-autoscaler awsRegion: us-east-1 autoDiscovery: clusterName: my-cluster image: repository: registry.k8s.io/autoscaling/cluster-autoscaler tag: v1.30.1 tolerances: - key: infra operator: Exists effect: NoSchedule nodeSelector: environment: "infra" rbac: create: true serviceAccount: name: cluster-autoscaler annotations: eks.amazonaws.com/role-arn: "role-aws" extraArgs: v: 1 stderrthreshold: info
With that configured and installed and our autoscaling groups created we have just enabled the automatic management of our nodes!
Why we decided to migrate to Karpenter
Our use case here at Projuris is as follows: we have a cluster development and another for production. After migrating to Gitlab SaaS we had a challenge of how to provision the r for the execution of our pipelines. It was decided that we would use the cluster development for provisioning these r. In the “first version” we opted for cluster auto scaler because it is a simpler configuration and already met our needs setup in production. But then we start to face some problems with this choice:
- Provisioning time: when starting a pipeline the machine provisioning time was a bit slow. The big point is that the cluster auto scaler pays a “toll” on cloud providers for provisioning a new urge.
- Difficulty in setting up groups: as we have some “profiles” of pipeline This management became a little complicated, because for each new profile a new node group needs to be created.
- Cost: to mitigate the slowness problem in startup of a new urge we had an “online” machine profile that was always on, even without running any pipeline.
What is Karpenter?
It is a solution of cluster autoscaling created by AWS, which promises the provisioning and consolidation of nodes always at the lowest possible cost. He is smart enough to know that for example, when buying a machine at AWS like on-demand, depending on the situation, is more affordable than if it were a machine spot. And that's just one of the features of this tool.
O Karpenter also works with the idea of “groups” of machines (which in the world of Karpenter we call it NodePools), but the difference is that we do this through CRDs (custom resource definitions) of the own Karpenter, that is, we have manifestos within our cluster with all these settings, eliminating the need for any node group created in AWS. Example of a NodePool No. Karpenter:
apiVersion: karpenter.sh/v1 kind: NodePool metadata: name: karpenter-gitlab-runner-small-online spec: template: metadata: labels: workload: gitlab-runners environment: karpenter-nodes-gitlab-runner-build-small-online spec: requirements: - key: "karpenter.sh/capacity-type" operator: In values: ["spot", “on-demand”] - key: "node.kubernetes.io/instance-type" operator: In values: ["m5d.large", "m5n.large", "m6id.large", "m6in.large"] nodeClassRef: group: karpenter.k8s.aws kind: EC2NodeClass name: my-node-class taints: - key: "gitlab-runner-karpenter" value: "true" effect: NoSchedule expirationAfter: Never disruption: consolidationPolicy: WhenEmpty consolidateAfter: 5m budgets: - nodes: "20%" limits: cpu: "500" memory: 500Gi
Besides the NodePool we need to create a NodeClass to define instance-specific details AWS:
apiVersion: karpenter.k8s.aws/v1 kind: EC2NodeClass metadata: name: my-node-class spec: amiFamily: AL2 role: "aws-role" tags: Name: nodes-k8s-nodes-gitlab-runner-small-online subnetSelectorTerms: - tags: karpenter.sh/subnet: "my-subnet" securityGroupSelectorTerms: - id: "sg-123" - id: "sg-456" - id: "sg-789" amiSelectorTerms: - id: "ami image" kubelet: clusterDNS: ["111.222.333.44"] blockDeviceMappings: - deviceName: /dev/xvda ebs: volumeSize: 40Gi volumeType: gp3 encrypted: true
NOTE: note that the name “my-node-class” needs to match the urge class configured in urge pool.
Like Karpenter helped us overcome the challenges presented?
- Provisioning time: as Karpenter talks directly to the APIs of cloud providers it is not necessary to pay the toll cluster auto scaler. We had a lot of problems with timeout in the provision of new nodes, after the exchange for Karpenter this problem simply disappeared precisely because provisioning is more efficient.
- Difficulty in setting up groups: with the solution of NodePools e NodeClass do Karpenter this configuration was trivial, and most importantly, versioned in our version control in Gitlab. In other words, you need to include a new machine profile in NodePool? No problem, just one c and Karpenter will already consider this in new provisions.
- Cost: We were able to use machines because now r with similar characteristics are allocated in nodes that support the required memory and CPU requirements. In other words, we are actually using all the computing power that that node provides. This also applies to the consolidation of nodes. As cluster auto scaler there were scripts complex to do the drain two nodes before consolidation. With the Karpenter this is configured in the NodePool in a very simplified way.
A great argument for management to justify investing in this type of change is cost. Below we have a cost comparison using the Cluster AutoScaler and Karpenter in January/25, where we achieved a total saving of 16%:
Figure 02: Period from 01/01 to 15/01 with ClusterAutoScaler
Figure 03: Period from 16/01 to 31/01 with the Karpenter
Final considerations
Migration to the Karpenter was a wise choice. We were able to simplify the management of our nodes with different profiles in a very simplified way. There is still room for some improvements, such as using a single NodePool to simplify even further, and let the r configure labels specific to the machine profile that should be provisioned for the runner (more in https://kubernetes.io/docs/reference/labels-annotations-taints/).
References
Karpenter (official doc): https://karpenter.sh/
Node Auto Scaling (official k8s doc): https://kubernetes.io/docs/concepts/cluster-administration/node-autoscaling/
