The HorizontalPodAutoscaler adjusts a scalable workload toward configured metric targets. Its output is a desired replica count, not a guarantee that additional Pods are already serving traffic. Scheduling, image startup, initialization, readiness, and available node capacity determine how quickly that recommendation becomes useful capacity. Design the control loop together with those delays.

The 300-second history contains the six-replica recommendation made at t=240 even though that step did not resize. At t=400 the new recommendation is two but the result stays six. At t=541 the high recommendation is 301 seconds old, so the model shrinks to two.
Source arithmetic and list-state model only. Historical Python-only execution remains passed; it did not exercise tolerance, rate policies, per-Pod handling, Kubernetes reconciliation or useful ready capacity. No new run was performed.

Metrics produce recommendations

For a simple per-Pod average, desired replicas are the ceiling of current replicas multiplied by current metric divided by target metric. A ratio of two suggests doubling before additional controls apply. A ratio below one can suggest shrinking, but tolerance, missing measurements, readiness handling, bounds, and scaling behavior can prevent or limit the action.

CPU utilization uses resource requests as its denominator. Changing requests can change utilization without changing consumed CPU, so coordinating vertical and horizontal scaling matters. The metrics API must be available and the target must support the scale subresource. HPA changes a Deployment’s count; the Deployment controller manages its ReplicaSets and Pods.

The default reconciliation interval is fifteen seconds, but that is not an end-to-end response promise. Metrics have their own collection windows and delays. Distinguish average usage across Pods from temporal averaging in the measurement pipeline, and distinguish both from stabilization of recommendations. A stale but valid number can mislead an otherwise healthy controller.

Stabilize recommendations, not the last resize

Downscale stabilization retains recent replica recommendations and selects the highest within the configured window. The default downscale window is five minutes; autoscaling/v2 behavior can customize it. This is not a simple cooldown measured from the last replica change. A high recommendation that left the count unchanged can still postpone a later shrink.

Rate policies separately bound how many Pods or what percentage may change over a period. Scale-up can therefore be limited too; it is not necessarily immediate or unlimited. Set a realistic floor for startup latency and a maximum consistent with capacity and cost. Stabilization reduces churn but does not compensate for a target that tracks the wrong bottleneck.

Choose pressure that extra replicas relieve

CPU can be useful for compute-bound requests, while queue depth or work per replica may better describe workers. Neither latency nor an arbitrary resource graph automatically makes a good control input. If all replicas wait on a saturated shared database, multiplying them can increase contention instead of reducing delay.

Evaluate what happens after one replica is added: can work move to it, does per-replica pressure fall, and does downstream capacity remain sufficient? Long-lived connections, uneven partitions, and cache warmup can all delay the intended effect. A controller cannot redistribute an application’s work simply by changing a number.

When it breaks

Missing metrics are unknown demand, not zero demand. With multiple metrics, a valid scale-up recommendation can still be used even if another source fails; a failed source prevents a shrink suggested by the remaining sources. Missing samples for individual Pods also receive conservative treatment. Use conditions and events to distinguish these cases instead of assuming every missing sample stops every action.

A maxReplicas ceiling can leave queues growing while ScalingLimited reports that the calculated recommendation was constrained. A high minimum can hide low demand. Repeated fixed replicas writes from deployment automation can fight HPA. Also inspect CPU request changes and startup readiness before blaming normal rolling updates: HPA supports Deployment rollouts and does not simply count every initializing Pod as zero-load capacity for unrestricted shrink.

Check it quickly

Save this as hpa_loop.py and run python3 hpa_loop.py. It is a standard-library model of ratio arithmetic, bounds, and a highest-recent-recommendation downscale window. It deliberately omits tolerance, rate policies, individual Pod samples, and live Kubernetes reconciliation.

import math

TARGET = 50.0
MIN_REPLICAS = 2
MAX_REPLICAS = 10
WINDOW_S = 300


def desired(replicas, value):
    raw = math.ceil(replicas * value / TARGET)
    return max(MIN_REPLICAS, min(MAX_REPLICAS, raw))


def step(replicas, history, now, value):
    if value is None:
        print(f"t={now}s metric=missing hold={replicas}")
        return replicas
    recommendation = desired(replicas, value)
    history[:] = [(t, count) for t, count in history if now - t <= WINDOW_S]
    history.append((now, recommendation))
    if recommendation < replicas:
        result = min(replicas, max(count for _, count in history))
    else:
        result = recommendation
    print(f"t={now}s raw={recommendation} {replicas}->{result}")
    return result


def main():
    history = []
    replicas = step(3, history, 0, 100)
    assert replicas == 6
    replicas = step(replicas, history, 240, 50)
    assert replicas == 6
    replicas = step(replicas, history, 400, 15)
    assert replicas == 6  # A recent recommendation, not a recent scale event.
    replicas = step(replicas, history, 541, 15)
    assert replicas == 2
    replicas = step(replicas, history, 600, None)
    assert replicas == 2
    assert desired(2, 500) == 10
    assert desired(2, 5) == 2
    print("PASS: recommendation history holds 6 at t=400, then shrinks to 2")


if __name__ == "__main__":
    main()

At time zero the model grows from three to six. At 240 seconds it recommends six again without resizing. At 400 seconds a new recommendation of two is held at six because the recent recommendation remains in the window, even though the last actual resize was more than five minutes ago. At 541 seconds that high recommendation has expired and the model shrinks to two.

Both language copies passed with Python 3.12.14 in this review, including missing-input hold and both bounds. Those assertions establish the model’s stated behavior only. For cluster tuning, compare real HPA conditions, recommendations, replica transitions, Pod readiness, and application latency over time. No live controller or metrics pipeline was exercised here.

Further reading

Sources