🎁 Bonus: Configurable HPA Tolerance

In real-world Kubernetes environments, application traffic doesn't always remain the same. CPU and memory usage can increase or decrease depending on application traffic and workload patterns.

Horizontal Pod Autoscaler (HPA) automatically adjusts the number of Pod replicas based on resource utilization or other configured metrics.

However, HPA doesn't trigger scaling for every small change in resource utilization. By default, HPA uses a 10% tolerance to avoid scaling Pods for small changes in CPU or memory usage.

For example, let's say you have an application running with an HPA configured to maintain 70% CPU utilization.

With the default 10% tolerance,

  • HPA considers scaling up when average CPU utilization goes above 77%.
  • HPA considers scaling down when average CPU utilization falls below 63%.
  • No scaling action is taken within the 63%–77% range due to tolerance.

This helps avoid unnecessary scaling when CPU usage keeps changing slightly.

However, in some applications, you may want HPA to respond to smaller increases in CPU usage while being more cautious when scaling down.

Previously, tolerance was configured at the cluster level, so you couldn't set different tolerance levels for individual HPA scale-up and scale-down operations.

This is where Configurable HPA Tolerance comes in.

HPA Tolerance Levels

HPA tolerance allows you to configure separate tolerance values for scaling up and scaling down.

Kubernetes introduced this feature in v1.33 and made it stable in v1.37.

We can configure tolerance using the behavior field in the HPA configuration.

Here is an example.

Let's understand the tolerance settings.

scaleUp: tolerance: 0.01 (1% tolerance) - This means your app will scale UP (add more pods) very aggressively.

For example, If your pods are running at 71% CPU, the HPA will immediately add more pods because it's above the 70.7% threshold.

scaleDown: tolerance: 0.05 (5% tolerance) - This means your app will scale DOWN (remove pods) more conservatively.

For example, if your pods are running at 67% CPU, the HPA will NOT remove pods because it's still above the 66.5% threshold.

Implementing HPA Tolerance Levels

For this example, we will deploy a simple Deployment that runs a CPU stress container and configure an HPA with custom tolerance values.

Before you start, make sure your cluster has a Metrics Server running.

First, deploy a simple deployment that runs a CPU stress container. Create a file named cpu-stress.yaml with the following contents.

Here, we are creating a Deployment with one replica that runs the stress-ng utility to stress the container. Now, create the Deployment.

Then, make sure the pod is up and running

Now let's create an HPA that scales the Deployment based on CPU utilization. Create a file named hpa-tolerance.yaml with the following contents.

In this HPA, we're configuring a CPU utilization target of 10%. Since the Deployment has a CPU request of 100m, 10% utilization represents 10m CPU usage.

We are also configuring a 2% scale-up tolerance and a 5% scale-down tolerance.

With this configuration, HPA can consider scaling up when average CPU utilization goes above 10.2%, which represents approximately 10.2m CPU. And HPA can consider scaling down when CPU utilization drops below 9.5%.

Now, apply the HPA configuration.

Once applied, verify that the HPA is created.

Now let's see how HPA responds to changes in CPU utilization. After some time, you may see the CPU percentage in HPA increase, and HPA scales the deployment.

As you can see, when the CPU utilization reaches 11%, which is above our configured scale-up threshold of 10.2%, HPA increases the number of Pods from one to two.

This is how custom tolerance allows HPA to respond to smaller changes in CPU utilization.

Complete and Continue