Autoscaling: Pods and Nodes
HPA, VPA and cluster autoscaling — what each solves, why CPU is often the wrong metric, and how to stop scale-up from crushing your database.
Three Axes
"Autoscaling" covers three different mechanisms solving three different problems:
| Mechanism | Changes | Answers |
|---|---|---|
| HPA | Number of pod replicas | "Do we need more instances?" |
| VPA | Requests and limits per pod | "Did we size each instance right?" |
| Cluster Autoscaler / Karpenter | Number of nodes | "Do we have anywhere to put them?" |
They compose. HPA adds pods; if no node has room, those pods sit Pending until node autoscaling provides capacity. Without the third, HPA silently stops working at the edge of your cluster.
The Prerequisite Everyone Misses
HPA scaling on CPU or memory needs metrics-server installed. It isn't part of a default cluster.
kubectl top nodes # if this errors, metrics-server is missingWithout it the HPA reports <unknown> for current utilisation and never scales:
kubectl get hpa
# NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS
# api Deployment/api <unknown>/70% 2 10 2<unknown> in the TARGETS column almost always means one of two things: metrics-server isn't installed, or the pods have no CPU requests. Utilisation is calculated as a percentage of the request — with no request there's no denominator, so the HPA has nothing to compute. This is where the resources guide from phase 4 becomes a hard dependency rather than a best practice.
A Basic HPA
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: api
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: api
minReplicas: 2
maxReplicas: 20
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70Use autoscaling/v2 — v1 supports only CPU.
The calculation is straightforward:
desiredReplicas = ceil(currentReplicas × (currentMetric / targetMetric))
4 replicas averaging 90% against a 70% target gives ceil(4 × 90/70) = 6.
CPU Is Often the Wrong Signal
CPU utilisation is the default and frequently a poor proxy for load.
Consider a worker consuming a queue. It spends most of its time waiting on I/O, so CPU stays at 20% while the queue grows to 50,000 messages. CPU-based scaling sees nothing wrong and never adds workers.
What you actually care about is queue depth, or request latency, or connections — the thing that reflects whether you're keeping up.
metrics:
- type: External
external:
metric:
name: sqs_queue_depth
selector:
matchLabels:
queue: jobs
target:
type: AverageValue
averageValue: "30" # aim for ~30 messages per replicaautoscaling/v2 supports three metric sources beyond Resource:
Pods— a per-pod metric averaged across replicas (requests per second).Object— a metric from another object (an Ingress's request rate).External— a metric from outside the cluster (queue depth, provider metrics).
These require a metrics adapter — commonly Prometheus Adapter, or KEDA, which is purpose-built for event-driven scaling and handles a long list of sources with much less wiring.
Ask what signal tells a human "we need more capacity". If the honest answer is "the queue is backing up" or "p99 latency is climbing", scale on that. CPU is the default because it's universally available, not because it's usually the best indicator.
Tuning Behaviour: the Thundering Herd
Default scaling reacts quickly upward and cautiously downward. The fast scale-up is what causes the roadmap's thundering-herd scenario: a traffic spike triggers a large batch of new pods at once, all of which open database connections simultaneously, and the database — already under the same spike — falls over.
The behavior field gives you control:
behavior:
scaleUp:
stabilizationWindowSeconds: 60
policies:
- type: Percent
value: 50 # add at most 50% more pods
periodSeconds: 60
- type: Pods
value: 4 # and never more than 4
periodSeconds: 60
selectPolicy: Min # apply the more conservative of the two
scaleDown:
stabilizationWindowSeconds: 300
policies:
- type: Percent
value: 25
periodSeconds: 60stabilizationWindowSeconds— consider recent recommendations rather than the instantaneous value, which prevents flapping on a spiky metric.- Policies cap the rate of change.
selectPolicy: Mintakes the most conservative applicable policy.
Rate-limiting scale-up is only half the fix for a thundering herd. The other half is a readiness probe that verifies the pod can actually reach its dependencies before it's added to the Service endpoints — the probes guide from phase 2. Otherwise new pods receive traffic the instant they start, before their connection pools are warm. A connection pooler in front of the database addresses the same problem from the other side.
Check yourself
An HPA on a queue worker is configured to scale on CPU at 70%. The queue grows steadily, jobs are delayed by hours, and the HPA never adds replicas. CPU sits at 20%. Why?
Vertical Pod Autoscaler
VPA adjusts requests and limits rather than replica count — the right tool when you sized a workload wrong rather than having too few of them.
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: api
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: api
updatePolicy:
updateMode: "Off" # recommend only, don't applyNever run VPA and HPA against the same resource metric on the same workload. They fight: HPA adds replicas because CPU is high, VPA raises CPU requests because usage is high, which lowers utilisation percentage, which makes HPA scale down — and the loop oscillates. Combining them is only safe when the HPA uses a different metric (a custom one such as queue depth) from the resource VPA manages.
updateMode: "Off" is genuinely useful on its own — VPA observes and writes recommendations without changing anything, giving you evidence-based numbers for the sizing exercise from phase 4.
Historically VPA had to recreate pods to change resources, which made it disruptive. In-place pod resizing improves this considerably by allowing resource changes without recreation; check what your cluster version supports.
Scaling the Cluster Itself
Pod autoscaling only works if there's somewhere to put the pods. When there isn't, pods go Pending with FailedScheduling — the filtering stage from the previous guide finding zero viable nodes.
Cluster Autoscaler watches for unschedulable pods and adds nodes from configured node groups, removing underutilised nodes when their pods can be placed elsewhere.
Karpenter takes a more flexible approach, provisioning right-sized instances directly based on what the pending pods actually need rather than scaling fixed node groups.
Either way, node provisioning takes time — often minutes. That latency is why minReplicas and some headroom matter: if you scale from zero capacity at the moment of a spike, you're waiting for a VM to boot while requests queue.
Two things commonly block node scale-down, and both are worth knowing because idle nodes are expensive: pods with no controller (a bare pod can't be safely moved), and restrictive PodDisruptionBudgets that prevent eviction. The next guide covers PDBs and this exact tension.
Diagnosing an HPA
kubectl get hpa api
kubectl describe hpa api # events explain scaling decisions and failures
kubectl top pods -l app=api # actual usagedescribe is where the answers are — it reports whether metrics are available, what it calculated, and why it did or didn't act.
Check yourself
Why shouldn't VPA and HPA both act on CPU for the same Deployment?
What's Next
Your workloads scale. The final guide of this phase covers surviving disruption — PodDisruptionBudgets, node drains, and the termination sequence where zero-downtime deploys are actually won or lost.