Running Stormforge with Karpenter and Kubecost

Michael Levan - Mar 11 - - Dev Community

As with all environments, including Dev, you want to ensure that whatever is running on said environment is operating at the utmost performance. Not only will it make the application stack more reliable, but it’s less calls at 2:00 AM for the engineers on-call and less support tickets from angry customers using your application.

In this blog post, you’ll learn about the three key ways to optimize performance across all fronts with Karpenter, Kubecost, and Stormforge.

Prerequisites

To follow along from a hands-on perspective with this blog post, you’ll need to sign up for the 30-day Stormforge trial. You can find a link to it here.

The Benefits Of All Three

When you think about performance optimization, you may be thinking “Why do I need three tools?”, and you’re absolutely right. Technically, you don’t. You could use Stormforge for all three needs (Stormforge can work directly with the Cluster Autoscaler). However, when implementing tools and platforms, the key is to ensure that what you’re implementing is the best for the job. That’s why implementing open-source tools like Kubecost and Karpenter to add value makes sense.

Let’s break down what each is for.

First, there’s Karpenter. Karpenter will be used for scaling nodes up and down in the most performant way possible. If you’re wondering why you wouldn’t just use Cluster Autoscaler because it’s built in, you absolutely can, but it’s slower. From a performance benchmark perspective, Karpenter wins in terms of getting nodes up and bringing them down faster.

Next, there's Stormforge. Stormforge will take care of the resource optimization for resources running on the cluster. It’ll manage the memory and CPU needs of the workloads. It can take care of all Requests and limits. Karpenter takes care of the infrastructure and Stormforge takes care of the workloads.

Lastly, there’s Kubecost. Kubecost takes care of… you guessed it, the cost. It ensures that cost optimization is properly implemented within your Kubernetes environment for both the cluster itself and the the workloads running within the cluster. It’ll give you suggestions on what can be optimized to save money and still perform as well as before.

Karpenter Configuration

The overall installation and configuration for Karpenter takes three key steps:

  1. Ensuring the proper IAM permissions.
  2. The installation of Karpenter itself.
  3. The configuration of Karpenter.

In this section, you’re going to learn how to set up all three to ensure Karpenter is properly running within your environment.

AWS Role

First things first - Karpenter needs permissions to scale up and scale down Worker Nodes. It doesn’t have the permission needed out of the box, so during the installation, you’ll point Karpenter to the proper IAM Role.

First, create a policy that uses the same config in the policy.json below.



{

"Statement": [
    {
        "Action": [
            "ssm:GetParameter",
            "iam:PassRole",
            "ec2:DescribeImages",
            "ec2:RunInstances",
            "ec2:DescribeSubnets",
            "ec2:DescribeSecurityGroups",
            "ec2:DescribeLaunchTemplates",
            "ec2:DescribeInstances",
            "ec2:DescribeInstanceTypes",
            "ec2:DescribeInstanceTypeOfferings",
            "ec2:DescribeAvailabilityZones",
            "ec2:DeleteLaunchTemplate",
            "ec2:CreateTags",
            "ec2:CreateLaunchTemplate",
            "ec2:CreateFleet",
            "ec2:DescribeSpotPriceHistory",
            "pricing:GetProducts"
        ],
        "Effect": "Allow",
        "Resource": "*",
        "Sid": "Karpenter"
    },
    {
        "Action": "ec2:TerminateInstances",
        "Condition": {
            "StringLike": {
                "ec2:ResourceTag/Name": "*karpenter*"
            }
        },
        "Effect": "Allow",
        "Resource": "*",
        "Sid": "ConditionalEC2Termination"
    }
],
"Version": "2012-10-17"
}
Enter fullscreen mode Exit fullscreen mode

This policy will ensure that Karpenter has the right permissions.

Next, create the policy.



aws iam create-policy --policy-name KarpenterControllerPolicy --policy-document file://policy.json
Enter fullscreen mode Exit fullscreen mode

After the policy is created, ensure that the OIDC provider is active and ready to use AWS IAM.



eksctl utils associate-iam-oidc-provider --cluster k8squickstart-cluster --approve
Enter fullscreen mode Exit fullscreen mode

Lastly, add the service account with the policy to your cluster.



eksctl create iamserviceaccount \
  --cluster "k8squickstart-cluster" --name karpenter --namespace karpenter \
  --role-name "KarpenterInstanceNodeRole" \
  --attach-policy-arn "arn:aws:iam::912101370089:policy/KarpenterControllerPolicy" \
  --role-only \
  --approve
Enter fullscreen mode Exit fullscreen mode

Installation

Now that the policy is ready, you can start the installation process for Karpenter.

First, configure a few environment various which contain:

  1. Your clusters name.
  2. The IAM role.
  3. The cluster endpoint

An example is below.



export CLUSTER_NAME="k8squickstart-cluster"
export KARPENTER_IAM_ROLE_ARN=
export CLUSTER_ENDPOINT=
Enter fullscreen mode Exit fullscreen mode

Next, install Karpenter with the associated environment variables.



helm upgrade --install --namespace karpenter --create-namespace \
karpenter karpenter/karpenter \
--set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \
--set clusterName=${CLUSTER_NAME} \
--set clusterEndpoint=${CLUSTER_ENDPOINT} \
--set aws.defaultInstanceProfile=KarpenterNodeInstanceProfile-${CLUSTER_NAME}
Enter fullscreen mode Exit fullscreen mode

Lastly, install the Karpenter CRD’s.



kubectl apply -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.sh_nodepools.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.sh_nodeclaims.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml
Enter fullscreen mode Exit fullscreen mode

You’re now ready for the Karpenter configuration.

Configuration

Once Karpenter is installed, you’ll then need to configure it for the Worker Nodes that you want it to manage as Karpenter doesn’t do this out of the box.

As an example, the configuration below sets up Karpenter to manage Linux-based nodes that are of instance type A or M.

Ensure that you specify the IAM Role name that you created for the config below.



kubectl apply -f - <<EOF
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["a", "m"]
        - key: karpenter.k8s.aws/instance-generation
          operator: Gt
          values: ["2"]
      nodeClassRef:
        name: default
  limits:
    cpu: 1000
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h # 30 * 24h = 720h
---
apiVersion: karpenter.k8s.aws/v1beta1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2
  role: "KarpenterNodeRole-k8squickstart-eks"
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: "k8squickstart-eks"
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: "k8squickstart-eks"
EOF
Enter fullscreen mode Exit fullscreen mode

You can then test out the configuration to ensure that proper scaling takes place.

Deploy a test workload.



kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: inflate
spec:
  replicas: 0
  selector:
    matchLabels:
      app: inflate
  template:
    metadata:
      labels:
        app: inflate
    spec:
      terminationGracePeriodSeconds: 0
      containers:
        - name: inflate
          image: public.ecr.aws/eks-distro/kubernetes/pause:3.7
          resources:
            requests:
              cpu: 1
EOF
Enter fullscreen mode Exit fullscreen mode

Scale the workload.



kubectl scale deployment inflate --replicas 5
Enter fullscreen mode Exit fullscreen mode

Check to ensure that Karpenter is working as expected.



kubectl logs -f -n "${KARPENTER_NAMESPACE}" -l app.kubernetes.io/name=karpenter -c controller
Enter fullscreen mode Exit fullscreen mode

Stormforge Configuration

Now that the Karpenter configuration is complete, you’ll implement the workload optimization piece, Stormforge. In the prerequisite section of this blog post, you went through the necessary prerequisites to sign up for Stormforge.

Once you’re signed in, you’ll see a screenshot similar to the one below.

Click the green + Add Cluster button.

Image description

Next, go through the steps of putting in information about your cluster.

Image description

You’ll reach a point where you need to download a values.yaml file which will contain the credentials Stormforge needs to have permissions to your Kubernetes cluster. Save the output in a values.yaml file.



clusterName: karpentertest
stormforge:
  address: https://api.stormforge.io/
authorization:
  issuer: https://api.stormforge.io/
  clientID: ef0a6fd
  clientSecret: AwTG
Enter fullscreen mode Exit fullscreen mode

The last step is to run the Stormforge Agent installation.



helm install stormforge-agent oci://registry.stormforge.io/library/stormforge-agent \
  --namespace stormforge-system \
  --create-namespace \
  --values values.yaml
Enter fullscreen mode Exit fullscreen mode

Kubecost Configuration

When implementing Kubecost, at first, you’ll most likely google “How do I install Kubecost?”. You’ll see several articles that show the installation, but the thing is it shows the installation with a Kubecost product key.

To get that product key, you have to contact Kubecost.

If you don’t want to do that, you can instead just install the 100% open-source version.

To install the open-source version, run the following Helm Chart.



helm upgrade --install kubecost \                            
  --repo https://kubecost.github.io/cost-analyzer/ cost-analyzer \
  --namespace kubecost --create-namespace
Enter fullscreen mode Exit fullscreen mode

You can ensure the resources are properly running with the command below.




kubectl get all -n kubecost
Enter fullscreen mode Exit fullscreen mode




Wrapping Up

Consolidating the tools you use into one usable and manageable stack is key for any implementation, and when it comes to overall performance optimization, this three-prong stack makes the most sense to ensure that the application stack you’re running is as efficient as possible.

Image
Why We Why We Lost Car Key Replacement (And You Should Also!)
Image
What To Focus On When Making Improvements To Automotive Key Repair
Image
A Trip Back In Time The Conversations People Had About Key Fob Repair Near Me 20 Years Ago
Image
30 Inspirational Quotes About Lost Car Key How To Replace
Image
5 Asbestos Lawsuit Projects For Any Budget
Image
Hacktoberfest 2024: My Journey and Key Learnings
Image
5 Arguments Filter Coffee Machine Is Actually A Good Thing
Image
Hands On Docker For Beginners Golang Dev

docker, backend, go, beginners
Image
Do You Know How To Explain Spare Car Key Price To Your Boss
Image
How Do I Get A Spare Car Key: What's No One Has Discussed
Image
How to Install a Cat Flap
Image
How To Get Assessed For ADHD: A Simple Definition
Image
5 Laws That Will Help The Mesothelioma And Asbestos Industry
Image
10 Places That You Can Find Coffee Filter Maker
Image
How to Create a Scroll Progress Bar in Next.js/React

nextjs, react, ui, ux
Image
10 Myths Your Boss Has About Asbestos Cancer Lawsuit Lawyer Mesothelioma Asbestos Cancer Lawsuit Lawyer Mesothelioma
Image
Mild Introduction to Modern Sequence Processing. Part 1: Looking into Recurrent Neural Networks
Image
17 Reasons Not To Beware Of Asbestos Attorney
Image
The Biggest Issue With Bi-Fold Door Repair And How You Can Solve It
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Terabox Video Player