Autoscaling Spring Boot on Kubernetes
The HorizontalPodAutoscaler adds and removes pods by comparing a metric with a target, and most examples use CPU. A JVM makes that metric unreliable: it burns CPU while starting and while the JIT compiler warms up, exactly when the autoscaler is deciding. This page measures that effect on a real Spring Boot service, runs the HPA's own formula against it, and shows the settings and the metrics that scale a Java service steadily.
Contents
How the HorizontalPodAutoscaler decides Why CPU misleads for a JVM What the HPA does with starting pods Warming up before the pod reports ready Tuning a CPU-based HPA for a JVM Scaling on request rate, backlog or latency KEDA: queue lag and Prometheus queries without an adapter Scale-down: stabilization and graceful shutdown Common pitfalls Which signal to use
How the HorizontalPodAutoscaler decides
A HorizontalPodAutoscaler (HPA) is a control loop in the kube-controller-manager
The core is one formula from the Kubernetes documentation:
desiredReplicas = ceil(currentReplicas × currentMetricValue / desiredMetricValue)
Two pods each using 140% of their CPU request, against a 70% target, give ceil(2 × 140 / 70) = 4behaviorminReplicasmaxReplicas
The table lists the five metric types; three APIs serve them:
| Type | Example | Served by | Best for |
|---|---|---|---|
Resource | Pod CPU as a percentage of its request | metrics.k8s.io | CPU-bound services with a steady cost per request |
ContainerResource | CPU of one container, ignoring sidecars | metrics.k8s.io | Pods with a service-mesh proxy or log shipper beside the application |
Pods | Requests per second per pod | custom.metrics.k8s.io | Web services, scaled on average load per pod |
Object | Requests per second on an Ingress | custom.metrics.k8s.io | Load measured at one shared object, such as an Ingress |
External | Kafka consumer lag, queue depth | external.metrics.k8s.io | Workers that drain a queue or topic |
The Kubernetes behaviour here was checked against the 1.33 and 1.34 documentation and source; the programs ran on Java 21 with Spring Boot 4.1. No cluster was available: the manifests were validated against the 1.33 and 1.34 schemas (and KEDA 2.21) but not applied, and a program implementing the HPA algorithm stands in for kubectl
Why CPU misleads for a JVM
CPU utilization works well as a scaling signal when each request costs about the same CPU at all times. A JVM, and the way Kubernetes measures its CPU, break that assumption:
- Startup: Spring Boot scans the classpath, loads thousands of classes and creates beans before it serves anything. A new pod uses nearly all the CPU it can get while this happens.
- JIT warm-up: the JVM interprets bytecode, then compiles hot methods with the fast C1 and later the optimizing C2 compiler. Compilation takes CPU and interpreted code is slower, so a fresh pod spends several times more CPU per request than a warm one.
- Garbage collection: GC threads use CPU that tracks allocation, not request count. A pod near its maximum heap collects more often, so its CPU rises without any rise in requests.
- The request is the denominator: utilization is usage divided by the CPU request. With a
request (a quarter of a core), a JVM reports 350% or more while it starts, because startup keeps about a full core busy.250m - Limits and throttling: a CPU limit is a CPU-time quota per 100 ms period. A JVM with many GC and compiler threads can spend it early in the period and then wait, which shows up as latency, not CPU.
To see the first two effects, this program starts a small Spring Boot service and drives it from the same JVM at a fixed 300 requests per second for one minute; each request builds and serializes a 40-line quote. It measures process CPU in 5-second windows, in millicores, minus the load generator's threads (JIT and GC work caused by the client still counts, so the figure is somewhat high):
package demo.warmup;
import java.lang.management.CompilationMXBean;
import java.lang.management.GarbageCollectorMXBean;
import java.lang.management.ManagementFactory;
import java.lang.management.ThreadInfo;
import java.lang.management.ThreadMXBean;
import java.math.BigDecimal;
import java.math.RoundingMode;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import com.sun.management.OperatingSystemMXBean;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.server.context.WebServerApplicationContext;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
@RestController
public class WarmupCpu {
record Line(String sku, int qty, BigDecimal price, BigDecimal total) { }
record Quote(String customer, List<Line> lines, BigDecimal net, BigDecimal tax) { }
// A small but real piece of work per request: build 40 lines, price them, serialize to JSON
@GetMapping("/quote")
Quote quote(@RequestParam("customer") String customer) {
List<Line> lines = new ArrayList<>();
BigDecimal net = BigDecimal.ZERO;
for (int i = 0; i < 40; i++) {
BigDecimal price = BigDecimal.valueOf(1999 + i * 37L, 2);
BigDecimal total = price.multiply(BigDecimal.valueOf(i % 5 + 1));
lines.add(new Line("SKU-" + i, i % 5 + 1, price, total));
net = net.add(total);
}
BigDecimal tax = net.multiply(new BigDecimal("0.2")).setScale(2, RoundingMode.HALF_UP);
return new Quote(customer, lines, net, tax);
}
static final OperatingSystemMXBean OS =
(OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean();
static final ThreadMXBean THREADS = ManagementFactory.getThreadMXBean();
static final CompilationMXBean JIT = ManagementFactory.getCompilationMXBean();
// CPU used by the load generator's own threads, so it can be taken out of the process total
static long clientCpuNanos() {
long sum = 0;
for (ThreadInfo t : THREADS.getThreadInfo(THREADS.getAllThreadIds())) {
if (t != null && (t.getThreadName().startsWith("load-")
|| t.getThreadName().startsWith("HttpClient"))) {
sum += Math.max(0, THREADS.getThreadCpuTime(t.getThreadId()));
}
}
return sum;
}
static long gcMillis() {
long sum = 0;
for (GarbageCollectorMXBean gc : ManagementFactory.getGarbageCollectorMXBeans()) {
sum += gc.getCollectionTime();
}
return sum;
}
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext ctx = SpringApplication.run(WarmupCpu.class,
"--server.port=0", "--spring.main.banner-mode=off", "--logging.level.root=WARN");
long startupCpuMs = OS.getProcessCpuTime() / 1_000_000;
long startupWallMs = ManagementFactory.getRuntimeMXBean().getUptime();
System.out.printf("CPUs available to the JVM: %d%n", Runtime.getRuntime()
.availableProcessors());
System.out.printf("Startup: %.1f CPU-seconds in %.1f s = %.2f cores on average%n",
startupCpuMs / 1000.0, startupWallMs / 1000.0,
(double) startupCpuMs / startupWallMs);
int port = ((WebServerApplicationContext) ctx).getWebServer().getPort();
URI uri = URI.create("http://localhost:" + port + "/quote?customer=acme");
AtomicInteger n = new AtomicInteger();
ExecutorService pool = Executors.newFixedThreadPool(8,
r -> new Thread(r, "load-" + n.incrementAndGet()));
HttpClient client = HttpClient.newBuilder().executor(pool).build();
HttpRequest request = HttpRequest.newBuilder(uri).build();
// Open-loop load: the same request rate every second, like steady production traffic
int perSecond = 300;
int windows = 12;
int windowSeconds = 5;
ScheduledExecutorService ticker = Executors.newSingleThreadScheduledExecutor();
AtomicInteger ok = new AtomicInteger();
ticker.scheduleAtFixedRate(() -> client
.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.thenAccept(r -> {
if (r.statusCode() == 200) {
ok.incrementAndGet();
}
}), 0, 1_000_000 / perSecond, TimeUnit.MICROSECONDS);
System.out.printf("Load: %d requests per second for %d s%n",
perSecond, windows * windowSeconds);
System.out.println("seconds service CPU (millicores) JIT ms GC ms requests");
List<Long> perWindow = new ArrayList<>();
long cpu0 = OS.getProcessCpuTime() - clientCpuNanos();
long jit0 = JIT.getTotalCompilationTime();
long gc0 = gcMillis();
int ok0 = 0;
for (int w = 0; w < windows; w++) {
Thread.sleep(windowSeconds * 1000L);
long cpu1 = OS.getProcessCpuTime() - clientCpuNanos();
long jit1 = JIT.getTotalCompilationTime();
long gc1 = gcMillis();
int ok1 = ok.get();
// CPU-milliseconds per second of wall time = millicores, as Kubernetes counts CPU
long milli = Math.round((cpu1 - cpu0) / 1e6 / windowSeconds / 10) * 10;
perWindow.add(milli);
System.out.printf("%3d-%-3d %25d %6d %5d %8d%n", w * windowSeconds,
(w + 1) * windowSeconds, milli, jit1 - jit0, gc1 - gc0, ok1 - ok0);
cpu0 = cpu1; jit0 = jit1; gc0 = gc1; ok0 = ok1;
}
ticker.shutdownNow();
pool.shutdown();
pool.awaitTermination(5, TimeUnit.SECONDS);
long first = perWindow.get(0);
double steady = perWindow.subList(windows - 3, windows).stream()
.mapToLong(Long::longValue).average().orElse(1);
System.out.printf("First 5 s / last 15 s: %.1fx%n", first / steady);
double[] q = new double[3];
for (int i = 0; i < 3; i++) {
q[i] = perWindow.subList(i * 3, i * 3 + 3).stream()
.mapToLong(Long::longValue).average().orElse(0) / steady;
}
System.out.printf("First three 15 s windows / last 15 s: %.1fx %.1fx %.1fx%n",
q[0], q[1], q[2]);
System.out.printf("Startup average / steady load: %.1fx%n",
(double) startupCpuMs / startupWallMs * 1000 / steady);
ctx.close();
}
}
CPUs available to the JVM: 2
Startup: 5.3 CPU-seconds in 2.9 s = 1.86 cores on average
Load: 300 requests per second for 60 s
seconds service CPU (millicores) JIT ms GC ms requests
0-5 1240 5717 83 1502
5-10 740 2885 56 1501
10-15 560 2177 39 1502
15-20 500 1740 28 1502
20-25 400 1209 28 1501
25-30 430 1349 23 1500
30-35 260 443 25 1501
35-40 210 197 28 1501
40-45 190 163 11 1501
45-50 230 381 24 1502
50-55 350 1068 30 1500
55-60 360 913 24 1502
First 5 s / last 15 s: 4.0x
First three 15 s windows / last 15 s: 2.7x 1.4x 0.7x
Startup average / steady load: 5.9x
Figures from this shared 2-core machine vary between runs, but the shape stayed the same:
- Startup used 0.9 to 1.9 cores on average, depending on what else the machine was doing: 180% to 380% utilization on a
request, and 5.9 times a warm pod's CPU in this run (500m line).Startup average - At a fixed 300 requests per second, the first 5 seconds cost 3.5 to 6 times the CPU of the last 15 seconds across runs (4.0x here). The same work on the same pod looks like a heavily loaded service, then like an idle one.
is compile time added across compiler threads, so it can exceed the 5,000 ms window. It falls in step with the CPU, which shows where the extra CPU went.JIT ms- CPU kept falling, with some noise, for most of the minute: C2 compiles a method only after thousands of invocations.
A CPU limit makes startup take longer. This program starts the same kind of service three times pinned with tasksetCPUs 0
package demo.startup;
import java.lang.management.GarbageCollectorMXBean;
import java.lang.management.ManagementFactory;
import java.util.ArrayList;
import java.util.List;
import com.sun.management.OperatingSystemMXBean;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
@RestController
public class StartupCpus {
@GetMapping("/hello")
String hello() {
return "hello";
}
public static void main(String[] args) throws Exception {
if (args.length > 0) {
child();
return;
}
String java = ProcessHandle.current().info().command().orElseThrow();
String cp = System.getProperty("java.class.path");
for (String cpus : List.of("0", "0,1")) {
long best = Long.MAX_VALUE;
String detail = "";
for (int run = 0; run < 3; run++) {
// taskset pins the JVM to the given CPUs, the way a CPU limit caps a container
List<String> cmd = new ArrayList<>(List.of("taskset", "-c", cpus, java,
"-cp", cp, StartupCpus.class.getName(), "child"));
Process p = new ProcessBuilder(cmd).redirectErrorStream(true).start();
String out = new String(p.getInputStream().readAllBytes()).trim();
p.waitFor();
String[] f = out.split(" ");
long ready = Long.parseLong(f[0]);
if (ready < best) {
best = ready;
detail = String.format("%.1f CPU-seconds, GC %s", Long.parseLong(f[1])
/ 1000.0, f[2]);
}
}
System.out.printf("CPUs %-4s ready after %.1f s (best of 3), %s%n",
cpus, best / 1000.0, detail);
}
}
// Starts the application, prints time to ready, CPU used and the collector, then exits
static void child() {
ConfigurableApplicationContext ctx = SpringApplication.run(StartupCpus.class,
"--server.port=0", "--spring.main.banner-mode=off", "--logging.level.root=WARN");
long ready = ManagementFactory.getRuntimeMXBean().getUptime();
OperatingSystemMXBean os =
(OperatingSystemMXBean) ManagementFactory.getOperatingSystemMXBean();
long cpu = os.getProcessCpuTime() / 1_000_000;
List<GarbageCollectorMXBean> gcs = ManagementFactory.getGarbageCollectorMXBeans();
String gc = gcs.get(0).getName().startsWith("G1") ? "G1" : "Serial";
System.out.println(ready + " " + cpu + " " + gc);
ctx.close();
}
}
CPUs 0 ready after 5.3 s (best of 3), 5.3 CPU-seconds, GC Serial
CPUs 0,1 ready after 3.1 s (best of 3), 5.6 CPU-seconds, GC G1
Both starts needed about the same CPU-seconds, so with half the CPUs the wall time was 1.7 times as long in this run, and with one CPU the JVM chose the Serial collector instead of G1 (see
What the HPA does with starting pods
The Kubernetes documentation names Java warm-up as the example of this problem. When the HPA scales on CPU, it sets some pods aside before averaging:
- Starting pods that are not ready: a pod whose readiness probe has not yet passed is first left out of the average. If the ready pods alone call for a scale-up, the HPA recomputes with each unready pod counted at 0% of its request, which makes the scale-up smaller rather than larger.
- The initialization period: for 5 minutes after a pod starts (
), a CPU sample counts only if the pod is ready and the whole sample window was collected after it became ready.--horizontal-pod-autoscaler-cpu-initialization-period - The readiness delay: after the initialization period, an unready pod is still set aside if its last readiness change came within 30 seconds of its start (
), which the controller takes to mean it has never been ready.--horizontal-pod-autoscaler-initial-readiness-delay
Both durations are cluster-wide kube-controller-manager
The simulator below (its source follows the results) implements the Kubernetes 1.33 controller's calculation: GetResourceReplicasWarmupCpu
== Ready after 30 s, JIT warm-up happens while serving traffic
time rps pods ready CPU seen counted recommended new count
1:00 1000 2 2 50% 50% 2 2
1:15 2400 2 2 120% 120% 4 4
1:30 2400 4 2 110% 120% 4 4
1:45 2400 4 2 110% 120% 4 4
2:00 2400 4 4 126% 126% 8 8
2:15 2400 8 4 89% 78% 8 8
2:30 2400 8 4 82% 63% 8 8
2:45 2400 8 8 63% 63% 8 8
3:00 2400 8 8 39% 39% 5 8
3:15 2400 8 8 32% 32% 4 8
3:30 2400 8 8 30% 30% 4 8
3:45 2400 8 8 30% 30% 4 8
7:45 2400 8 8 30% 30% 4 5
8:00 2400 5 5 48% 48% 4 4
pods started: 6, peak replicas: 8, replicas at the end: 4
== Same, with scaleUp.stabilizationWindowSeconds: 60
time rps pods ready CPU seen counted recommended new count
1:00 1000 2 2 50% 50% 2 2
1:15 2400 2 2 120% 120% 4 2
1:30 2400 2 2 120% 120% 4 2
1:45 2400 2 2 120% 120% 4 2
2:00 2400 2 2 120% 120% 4 4
2:15 2400 4 2 110% 120% 4 4
2:30 2400 4 2 110% 120% 4 4
2:45 2400 4 4 126% 126% 8 4
3:00 2400 4 4 78% 78% 5 4
3:15 2400 4 4 63% 63% 4 4
3:30 2400 4 4 60% 60% 4 4
3:45 2400 4 4 60% 60% 4 4
pods started: 2, peak replicas: 4, replicas at the end: 4
== Application warms itself up before reporting ready (ready after 75 s)
time rps pods ready CPU seen counted recommended new count
1:00 1000 2 2 50% 50% 2 2
1:15 2400 2 2 120% 120% 4 4
1:30 2400 4 2 110% 120% 4 4
1:45 2400 4 2 110% 120% 4 4
2:00 2400 4 2 110% 120% 4 4
2:15 2400 4 2 110% 120% 4 4
2:30 2400 4 2 110% 120% 4 4
2:45 2400 4 4 60% 60% 4 4
3:00 2400 4 4 60% 60% 4 4
3:15 2400 4 4 60% 60% 4 4
3:30 2400 4 4 60% 60% 4 4
3:45 2400 4 4 60% 60% 4 4
pods started: 2, peak replicas: 4, replicas at the end: 4
== Swinging traffic, scaleDown.stabilizationWindowSeconds: 0
pods started: 10, peak replicas: 4, replicas at the end: 2
replicas at each minute: 4 4 2 2 4 4 2 2 4 4 2 2 4 4 2 2 4 4 2 2
== Swinging traffic, scaleDown.stabilizationWindowSeconds: 300 (default)
pods started: 2, peak replicas: 4, replicas at the end: 4
replicas at each minute: 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
Each row is one HPA evaluation; after 3:45, only rows that change the count are printed. The rpspodsreadyCPU seencountedrecommendednew count
- In the first scenario the HPA correctly asks for 4 pods at 1:15. While the two new pods start, it sets them aside: at 1:30
is 110%, but the recommendation stays at 4.CPU seen - At 2:00 the new pods have been ready for one full window and are counted, while their JIT compilers are still busy. Counted CPU reads 126%, and
, twice what the traffic needs. The scale-down window keeps the highest recommendation of the last 5 minutes, so the 8 pods run for five and a half minutes, more than four of them at 30%, until the count falls to 5 at 7:45 and to 4 at 8:00.ceil(4 × 126 / 70) = 8 - A 60-second scale-up stabilization window acts on the lowest recommendation of the last minute. At 2:45 the formula asks for 8 again, but the window still holds the recommendations of 4 made while the new pods were unready, so the count stays at 4; a window shorter than the warm-up would miss this. The price: the genuine scale-up waits 45 seconds longer.
- In the third scenario the application warms itself up before it reports ready, and the simulation treats it as fully warm from then on. The HPA never counts a cold pod, and the count goes straight to 4. A real warm-up removes most of the cold cost but not all, so a scale-up window is still worth keeping.
The simulation is deliberately simple: pods start when created, traffic spreads evenly, and CPU is a fixed cost per request. Image pulls and scheduling make the spike land later, not smaller, and a rolling update brings in cold pods too, so a CPU-based HPA can scale up during every rollout. The source:
import java.util.ArrayList;
import java.util.List;
import java.util.function.IntUnaryOperator;
public class HpaSimulator {
static final int STEP = 15; // HPA sync period and metrics window, seconds
static final double REQUEST_M = 1000; // CPU request per pod, millicores
static final double TARGET = 0.70; // averageUtilization: 70
static final double TOLERANCE = 0.10; // default cluster-wide tolerance
static final int CPU_INIT_PERIOD = 300; // cpu-initialization-period flag, 5 minutes
static final int MIN = 2;
static final int MAX = 20;
// CPU per request after the pod becomes ready, relative to a warm JVM, per 15 s window.
// From a WarmupCpu run, 15 s averages against the last 15 s: 3.2x, 1.6x, 1.1x, then warm.
static final double[] COLD = {3.2, 1.6, 1.1};
record Pod(int start, int readyAt, boolean warmBeforeReady) { }
record Rec(int time, int replicas) { }
static double factor(Pod p, int t) {
if (p.warmBeforeReady()) {
return 1.0;
}
int w = (t - p.readyAt()) / STEP;
return w < COLD.length ? COLD[w] : 1.0;
}
/** Runs one scenario and prints what the HPA saw and decided at each evaluation. */
static void run(String title, IntUnaryOperator rps, int seconds, int readySeconds,
boolean warmBeforeReady, int upWindow, int downWindow, boolean verbose) {
System.out.println("== " + title);
List<Pod> pods = new ArrayList<>();
pods.add(new Pod(-600, -570, warmBeforeReady));
pods.add(new Pod(-600, -570, warmBeforeReady));
List<Rec> history = new ArrayList<>();
int started = 0;
int peak = pods.size();
StringBuilder timeline = new StringBuilder();
if (verbose) {
System.out.println(" time rps pods ready CPU seen counted"
+ " recommended new count");
}
for (int t = 0; t < seconds; t += STEP) {
final int now = t;
List<Pod> ready = pods.stream().filter(p -> p.readyAt() <= now).toList();
double perPod = (double) rps.applyAsInt(t) / ready.size();
// CPU each pod used during [t, t + STEP): a pod that is still starting burns
// a full core, which is 100% of its 1000m request
double sumAll = 0;
double sumCounted = 0;
int counted = 0;
boolean anyUnready = false;
for (Pod p : pods) {
double cpu = p.readyAt() <= t ? perPod * factor(p, t) : 1000;
sumAll += cpu;
int sampleEnd = t + STEP;
// groupPods(): inside the initialization period a sample counts only if the
// pod is ready and the whole window was collected after it became ready
boolean unready = sampleEnd - p.start() < CPU_INIT_PERIOD
? p.readyAt() > sampleEnd || sampleEnd < p.readyAt() + STEP
: p.readyAt() > sampleEnd;
if (unready) {
anyUnready = true;
} else {
sumCounted += cpu;
counted++;
}
}
int current = pods.size();
double ratio = sumCounted / (counted * REQUEST_M) / TARGET;
int desired;
if (!(anyUnready && ratio > 1) && Math.abs(ratio - 1) <= TOLERANCE) {
desired = current;
} else if (!(anyUnready && ratio > 1)) {
desired = (int) Math.ceil(ratio * counted);
} else {
// scale-up with unready pods: assume they use 0% and recompute
double newRatio = sumCounted / (current * REQUEST_M) / TARGET;
desired = newRatio <= 1 + TOLERANCE ? current : (int) Math.ceil(newRatio * current);
}
history.add(new Rec(t + STEP, desired));
// stabilization: lowest recommendation in the up window, highest in the down window
int up = desired;
int down = desired;
for (Rec r : history) {
if (r.time() > t + STEP - upWindow) {
up = Math.min(up, r.replicas());
}
if (r.time() > t + STEP - downWindow) {
down = Math.max(down, r.replicas());
}
}
int next = Math.min(Math.max(current, up), down);
next = Math.min(next, Math.max(current + 4, current * 2)); // default scale-up policy
next = Math.max(MIN, Math.min(MAX, next)); // then minReplicas and maxReplicas
// from 1:00 print every evaluation up to 3:45, then only those that change the count
if (verbose && ((t + STEP >= 60 && t + STEP <= 225) || next != current)) {
System.out.printf("%6s %5d %4d %5d %7.0f%% %6.0f%% %11d %9d%n",
clock(t + STEP),
rps.applyAsInt(t), current, ready.size(),
sumAll / (current * REQUEST_M) * 100,
counted == 0 ? 0 : sumCounted / (counted * REQUEST_M) * 100, desired, next);
}
while (pods.size() < next) {
pods.add(new Pod(t + STEP, t + STEP + readySeconds, warmBeforeReady));
started++;
}
while (pods.size() > next) {
pods.removeLast();
}
peak = Math.max(peak, pods.size());
if ((t + STEP) % 60 == 0) {
timeline.append(pods.size()).append(' ');
}
}
System.out.printf("pods started: %d, peak replicas: %d, replicas at the end: %d%n",
started, peak, pods.size());
if (!verbose) {
System.out.println("replicas at each minute: " + timeline.toString().trim());
}
System.out.println();
}
static String clock(int s) {
return String.format("%d:%02d", s / 60, s % 60);
}
public static void main(String[] args) {
// Traffic jumps from 1,000 to 2,400 requests/s at 1:00. A warm pod uses 1 millicore
// per request/s, so 2,400 requests/s need 2,400 / 700 = 3.4, that is 4 pods at 70%.
IntUnaryOperator step = t -> t < 60 ? 1000 : 2400;
run("Ready after 30 s, JIT warm-up happens while serving traffic",
step, 600, 30, false, 0, 300, true);
run("Same, with scaleUp.stabilizationWindowSeconds: 60",
step, 600, 30, false, 60, 300, true);
run("Application warms itself up before reporting ready (ready after 75 s)",
step, 600, 75, true, 0, 300, true);
// Traffic that swings between 1,200 and 2,800 requests/s every 2 minutes, for 20 min
IntUnaryOperator wave = t -> (t / 120) % 2 == 0 ? 2800 : 1200;
run("Swinging traffic, scaleDown.stabilizationWindowSeconds: 0",
wave, 1200, 30, true, 0, 0, false);
run("Swinging traffic, scaleDown.stabilizationWindowSeconds: 300 (default)",
wave, 1200, 30, true, 0, 300, false);
}
}
Warming up before the pod reports ready
After the context has started, Spring Boot sets liveness to CORRECTApplicationRunnerCommandLineRunnerACCEPTING_TRAFFIC
package demo.readywarm;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.ArrayList;
import java.util.List;
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.server.context.WebServerApplicationContext;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
@RestController
public class WarmupBeforeReady implements ApplicationRunner {
record Item(String sku, int qty, long cents) { }
private final WebServerApplicationContext web;
WarmupBeforeReady(WebServerApplicationContext web) {
this.web = web;
}
@GetMapping("/quote")
List<Item> quote(@RequestParam("customer") String customer) {
List<Item> items = new ArrayList<>();
for (int i = 0; i < 40; i++) {
items.add(new Item(customer + "-" + i, i % 5 + 1, 1999L + i * 37L));
}
return items;
}
// Runners run after the web server has started but before Spring Boot publishes
// ReadinessState.ACCEPTING_TRAFFIC, so the readiness probe fails while this runs
@Override
public void run(ApplicationArguments args) throws Exception {
String base = "http://localhost:" + web.getWebServer().getPort();
HttpClient client = HttpClient.newHttpClient();
System.out.println("During warm-up, liveness: " + get(client, base
+ "/actuator/health/liveness"));
System.out.println("During warm-up, readiness: " + get(client, base
+ "/actuator/health/readiness"));
HttpRequest quote = HttpRequest.newBuilder(URI.create(base + "/quote?customer=w"))
.build();
List<Long> batches = new ArrayList<>();
for (int batch = 0; batch < 6; batch++) {
long start = System.nanoTime();
for (int i = 0; i < 500; i++) {
client.send(quote, HttpResponse.BodyHandlers.discarding());
}
batches.add(System.nanoTime() - start);
}
long total = batches.stream().mapToLong(Long::longValue).sum();
System.out.printf("Warm-up: 6 batches of 500 requests in %.1f s, last batch %.1fx "
+ "faster than the first%n", total / 1e9, (double) batches.get(0) / batches.get(5));
}
static String get(HttpClient client, String url) throws Exception {
HttpResponse<String> r = client.send(HttpRequest.newBuilder(URI.create(url)).build(),
HttpResponse.BodyHandlers.ofString());
return r.statusCode() + " " + r.body();
}
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext ctx = SpringApplication.run(WarmupBeforeReady.class,
"--server.port=0", "--spring.main.banner-mode=off", "--logging.level.root=WARN");
String base = "http://localhost:"
+ ((WebServerApplicationContext) ctx).getWebServer().getPort();
System.out.println("After startup, readiness: " + get(HttpClient.newHttpClient(),
base + "/actuator/health/readiness"));
ctx.close();
}
}
During warm-up, liveness: 200 {"status":"UP"}
During warm-up, readiness: 503 {"status":"OUT_OF_SERVICE"}
Warm-up: 6 batches of 500 requests in 4.6 s, last batch 3.1x faster than the first
After startup, readiness: 200 {"status":"UP"}
- During the runner, liveness answers
(theUP state), so Kubernetes does not restart the pod; readiness answers 503CORRECT , so the pod gets no traffic and the HPA sets its CPU aside.OUT_OF_SERVICE - The last batch of 500 requests ran 3.1 times faster than the first in this run, and between 2.5 and 3.4 times across runs. After
returns, readiness turnsrun and the pod joins the Service warm.UP
In Spring Boot 4 the liveness and readiness groups are on by default (management.endpoint.health.probes.enabledtrue
The Deployment needs probes that allow for the extra time, and a CPU request that matches the warm load:
# deployment.yaml: settings that matter for autoscaling
apiVersion: apps/v1
kind: Deployment
metadata:
name: payment-service
namespace: payments
spec:
# no replicas field: the HPA owns the count
selector:
matchLabels:
app: payment-service
template:
metadata:
labels:
app: payment-service
spec:
terminationGracePeriodSeconds: 45
containers:
- name: app
image: registry.example.com/payment-service:4f2a91c
ports:
- containerPort: 8080
resources:
requests:
cpu: "1" # the HPA's 100%: size it for warm, steady load
memory: 2Gi
limits:
memory: 2Gi # no CPU limit, so startup is not throttled
env:
- name: JAVA_TOOL_OPTIONS
value: "-XX:ActiveProcessorCount=2 -XX:+UseG1GC -XX:MaxRAMPercentage=75"
startupProbe: # allows 60 s for startup before liveness takes over
httpGet:
path: /actuator/health/liveness
port: 8080
periodSeconds: 5
failureThreshold: 12
readinessProbe: # fails until the warm-up runner has finished
httpGet:
path: /actuator/health/readiness
port: 8080
periodSeconds: 5
livenessProbe:
httpGet:
path: /actuator/health/liveness
port: 8080
periodSeconds: 10
lifecycle:
preStop:
sleep:
seconds: 10 # keep serving while endpoints are removed
- No
field: with an HPA in charge, everyreplicas of a manifest that setskubectl apply resets the count. On an existing Deployment, deleting the field and running a client-sidereplicas drops the count to 1 for a moment; remove it first withkubectl apply , or use server-side apply.kubectl apply edit-last-applied - CPU request: this is what 100% utilization means to the HPA. Set it to what a warm pod uses at the load you want each pod to carry, not to the startup peak.
- No CPU limit: the pod may borrow idle CPU while it starts and compiles. The JVM then sees every CPU on the node, so
caps the GC, JIT and common-pool threads it sizes from the CPU count (2 because the pod may exceed its request). Check that no namespace LimitRange adds a default CPU limit.-XX:ActiveProcessorCount=2 - Memory and collector: the JVM also picks Serial when it sees less than 1,792 MB, so a 1Gi limit gives Serial even with two processors;
makes the choice explicit. The 2Gi limit leaves a quarter outside the 75% heap for metaspace, thread stacks, code cache and direct buffers.-XX:+UseG1GC - Probes: the startup probe allows 60 seconds; readiness waits for the warm-up runner.
Kubernetes probes for Spring Boot covers the timing. - preStop and grace period: see Scale-down below.
Tuning a CPU-based HPA for a JVM
If CPU is the signal you have, these settings address the overshoot shown in the simulation:
# hpa-cpu.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: payment-service
namespace: payments
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: payment-service
minReplicas: 2
maxReplicas: 20
metrics:
- type: ContainerResource
containerResource:
name: cpu
container: app # the JVM only, not sidecars
target:
type: Utilization
averageUtilization: 70
behavior:
scaleUp:
stabilizationWindowSeconds: 60 # act on the lowest recommendation of the last minute
policies:
- type: Percent
value: 100 # at most double per minute; replaces the default add-4
periodSeconds: 60
scaleDown:
stabilizationWindowSeconds: 300 # act on the highest recommendation of the last 5 min
policies:
- type: Pods
value: 2 # remove at most 2 pods per minute
periodSeconds: 60
- Scale-up stabilization window: the default is 0, so a single high reading scales up at once. Make the window a little longer than your service's warm-up after readiness: 60 seconds fit the 45 seconds simulated above, but a service with many endpoints can take minutes. Measure yours by starting one pod under steady load and watching its CPU until it stops falling. A longer window reacts later to a real surge.
- Scale-up policy: the default allows doubling, or adding 4 pods, every 15 seconds. Listing policies replaces both defaults, so this one adds only 2 pods per minute at 2 replicas; add a
policy if small Deployments must grow faster (the policy that allows the most change wins). A slower rate gives new pods time to warm up.Pods - Target utilization: 60% to 75% leaves headroom while new pods start and warm up; the longer the startup, the lower the target.
- ContainerResource: with a sidecar in the pod,
divides the total usage of all containers by their total requests. Naming the application container keeps a busy proxy from hiding an idle JVM, or the reverse.Resource
Do not scale a JVM on memory. The heap grows to what the collector finds convenient and is given back slowly, if at all, so after a burst the HPA may not scale down for a long time.
Scaling on request rate, backlog or latency
A metric that counts the work itself does not overshoot during warm-up: a cold pod and a warm pod receive the same requests per second, though the cold one uses several times the CPU. Spring Boot Actuator records such metrics through Micrometer. This program serves two endpoints, registers a gauge for a work backlog, sends 25 GET and 7 POST requests, and prints the matching lines from /actuator/prometheus
package demo.hpametrics;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import io.micrometer.core.instrument.Gauge;
import io.micrometer.core.instrument.MeterRegistry;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.web.server.context.WebServerApplicationContext;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RestController;
@SpringBootApplication
@RestController
public class HpaMetrics {
// Work accepted but not yet processed: the backlog a worker deployment would scale on
private final BlockingQueue<String> backlog = new LinkedBlockingQueue<>();
HpaMetrics(MeterRegistry registry) {
Gauge.builder("orders.backlog", backlog, BlockingQueue::size)
.description("Orders waiting to be processed")
.register(registry);
}
@PostMapping("/orders")
String accept() {
backlog.add("order");
return "accepted";
}
@GetMapping("/orders/{id}")
String find(@PathVariable("id") String id) {
return "order " + id;
}
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext ctx = SpringApplication.run(HpaMetrics.class,
"--server.port=0", "--spring.main.banner-mode=off", "--logging.level.root=WARN",
"--management.endpoints.web.exposure.include=prometheus",
"--management.metrics.distribution.slo.http.server.requests=250ms,1s",
"--management.metrics.tags.application=orders");
int port = ((WebServerApplicationContext) ctx).getWebServer().getPort();
String base = "http://localhost:" + port;
HttpClient client = HttpClient.newHttpClient();
// one scrape first, so the first measured request is not the one that starts MVC
send(client, HttpRequest.newBuilder(URI.create(base + "/actuator/prometheus")).build());
for (int i = 0; i < 25; i++) {
send(client, HttpRequest.newBuilder(URI.create(base + "/orders/" + i)).build());
}
for (int i = 0; i < 7; i++) {
send(client, HttpRequest.newBuilder(URI.create(base + "/orders"))
.POST(HttpRequest.BodyPublishers.noBody()).build());
}
String scrape = client.send(
HttpRequest.newBuilder(URI.create(base + "/actuator/prometheus")).build(),
HttpResponse.BodyHandlers.ofString()).body();
System.out.println("Sent 25 GET /orders/{id} and 7 POST /orders. Scrape, filtered:");
for (String line : scrape.split("\n")) {
boolean wanted = line.startsWith("orders_backlog")
|| line.startsWith("# TYPE orders_backlog")
|| line.startsWith("# TYPE http_server_requests_seconds ")
|| line.startsWith("http_server_requests_active_seconds_gcount{")
|| (line.startsWith("http_server_requests_seconds_")
&& line.contains("uri=\"/orders"));
// the _sum and _max series hold measured times, which differ on every run
if (wanted && !line.contains("_sum{") && !line.contains("_max{")) {
System.out.println(line);
}
}
ctx.close();
}
static void send(HttpClient client, HttpRequest request) throws Exception {
client.send(request, HttpResponse.BodyHandlers.discarding());
}
}
Sent 25 GET /orders/{id} and 7 POST /orders. Scrape, filtered:
http_server_requests_active_seconds_gcount{application="orders",exception="none",method="GET",outcome="SUCCESS",status="200",uri="UNKNOWN"} 1
http_server_requests_active_seconds_gcount{application="orders",exception="none",method="POST",outcome="SUCCESS",status="200",uri="UNKNOWN"} 0
# TYPE http_server_requests_seconds histogram
http_server_requests_seconds_bucket{application="orders",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/orders/{id}",le="0.25"} 25
http_server_requests_seconds_bucket{application="orders",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/orders/{id}",le="1.0"} 25
http_server_requests_seconds_bucket{application="orders",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/orders/{id}",le="+Inf"} 25
http_server_requests_seconds_count{application="orders",error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/orders/{id}"} 25
http_server_requests_seconds_bucket{application="orders",error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/orders",le="0.25"} 7
http_server_requests_seconds_bucket{application="orders",error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/orders",le="1.0"} 7
http_server_requests_seconds_bucket{application="orders",error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/orders",le="+Inf"} 7
http_server_requests_seconds_count{application="orders",error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/orders"} 7
# TYPE orders_backlog gauge
orders_backlog{application="orders"} 7.0
is a counter per endpoint, method and status: 25 forhttp_server_requests_seconds_count and 7 forGET /orders/{id} . Prometheus'sPOST /orders turns it into requests per second, usually the best scaling metric for a web service. Therate() label is the route template, so the 25 ids share one series.uri- The
andle="0.25" buckets come from thele="1.0" property: they count requests that took at most 250 ms and at most 1 s, enough to compute the share of slow requests.slo is the number of requests in flight, from a long task timer (hencehttp_server_requests_active_seconds_gcount ). Its_gcount isuri because it is tagged before Spring has matched a route; the one activeUNKNOWN is the scrape itself. It suits slow, blocking endpoints, where CPU stays low while threads wait.GET is the custom gauge: 7 orders accepted and not yet processed. Workers scale on this kind of number. In production, keep the backlog in a broker or database: a gauge published by the workers disappears when they scale to zero, and nothing scales them back up.orders_backlog- The
tag comes fromapplication="orders" and keeps services apart in a shared Prometheus.management.metrics.tags.application
The program needs spring-boot-starter-actuatormicrometer-registry-prometheus
Latency makes a poor scaling target: it rises for reasons more pods do not fix, such as a slow database, and more pods add connections to the bottleneck. Scale on request rate and alert on latency. A slow dependency raises in-flight requests too, so scale on them only where the slowness is the pod's own work.
The HPA cannot query Prometheus itself. It reads the custom.metrics.k8s.iorules.customnamespacepod
# prometheus-adapter rules
rules:
- seriesQuery: 'http_server_requests_seconds_count{namespace!="",pod!=""}'
resources:
overrides:
namespace: {resource: "namespace"}
pod: {resource: "pod"}
name:
matches: "^http_server_requests_seconds_count$"
as: "http_server_requests_per_second"
metricsQuery: >-
sum(rate(<<.Series>>{<<.LabelMatchers>>,uri!~"/actuator.*"}[2m]))
by (<<.GroupBy>>)
The adapter fills in the series name, the namespace and pod selectors, and the pod label. The uri!~"/actuator.*"
# hpa-requests.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: payment-service
namespace: payments
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: payment-service
minReplicas: 2
maxReplicas: 20
metrics:
- type: Pods
pods:
metric:
name: http_server_requests_per_second
target:
type: AverageValue
averageValue: "400" # 70-80% of what one warm pod handles at acceptable latency
- type: ContainerResource # safety net; the higher recommendation wins
containerResource:
name: cpu
container: app
target:
type: Utilization
averageUtilization: 150 # well above warm use at 400 requests/s
behavior:
scaleDown:
stabilizationWindowSeconds: 300
With several metrics, the HPA computes a replica count for each and uses the largest. The CPU target must sit clearly above what a warm pod uses at 400 requests per second, or CPU decides every scale-up, cold pods included. With the request sized for that load, warm use is near 100%, hence 150 (allowed without a CPU limit). The safety net then acts only if requests become much more expensive, for example after a slower release. Find the per-pod rate by load-testing one warm pod until latency or errors exceed what you accept, and take 70% to 80% of that rate.
KEDA: queue lag and Prometheus queries without an adapter
KEDA is a CNCF project that serves external.metrics.k8s.ioScaledObject
# scaledobject-requests.yaml: the same request-rate scaling, with KEDA
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: payment-service
namespace: payments
spec:
scaleTargetRef:
name: payment-service
minReplicaCount: 2
maxReplicaCount: 20
advanced:
horizontalPodAutoscalerConfig:
behavior: # passed through to the HPA KEDA creates
scaleDown:
stabilizationWindowSeconds: 300
triggers:
- type: prometheus
metadata:
serverAddress: http://prometheus.monitoring.svc:9090
query: >-
sum(rate(http_server_requests_seconds_count{namespace="payments",
application="payment-service",uri!~"/actuator.*"}[2m]))
threshold: "400" # requests/s per pod
Use this ScaledObject in place of hpa-requests.yamlAmbiguousSelectorapplicationmanagement.metrics.tags.applicationAverageValue
For a Kafka consumer, such as one built with
# scaledobject-kafka.yaml: scale consumers on lag
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
name: order-consumer
namespace: payments
spec:
scaleTargetRef:
name: order-consumer
pollingInterval: 30
minReplicaCount: 1
maxReplicaCount: 12 # no more than the topic's partitions
advanced:
horizontalPodAutoscalerConfig:
behavior:
scaleUp:
stabilizationWindowSeconds: 60 # new consumers start slowly
triggers:
- type: kafka
metadata:
bootstrapServers: kafka.kafka.svc:9092
consumerGroup: order-consumer
topic: orders
lagThreshold: "500" # target lag per replica
- Partitions cap the useful replicas: each partition is read by only one consumer in a group, so extra pods sit idle. KEDA's Kafka scaler stops at the partition count unless
is set.allowIdleConsumers - Every scale event is a rebalance: the group reassigns partitions and consumption pauses meanwhile. A scale-down stabilization window keeps a fluctuating lag from causing a rebalance every minute.
- New consumers start slowly: a new consumer pod processes slowly for its first seconds, so lag may keep rising briefly after a scale-up; the scale-up window in the example covers that.
- KEDA's
(300 seconds by default) only governs the last step to zero replicas, withcooldownPeriod . Between 1 and N replicas, the HPA's ownminReplicaCount: 0 settings apply.behavior - Scale from zero is a cold start: the first message waits for the next poll (
, 30 seconds by default), scheduling, JVM startup and warm-up. An activation threshold (pollingInterval for Kafka) sets how much lag wakes the workload.activationLagThreshold
Scale-down: stabilization and graceful shutdown
The last two scenarios of the simulation show why the default 5-minute scale-down window matters, with traffic swinging between 1,200 and 2,800 requests per second every two minutes:
- With
, the count followed every swing, 4, 2, 4, 2, and started 10 pods in 20 minutes, each paying for a JVM startup and warm-up. These simulated pods warm up before they are ready; without that, each would also report high CPU while warming, which can trigger the next scale-up.scaleDown.stabilizationWindowSeconds: 0 - With the default 300 seconds, the HPA kept the highest recommendation of the last 5 minutes and stayed at 4 pods throughout, starting only the 2 pods the first peak needed.
Churn costs Java services more. Among otherwise equal pods, the ReplicaSet controller removes the ones that became ready most recently first (on a logarithmic time scale), so pods that have just finished their warm-up usually go first. A removed pod may also be in the middle of requests or messages. Each deleted pod goes through the same sequence as in a rolling update:
- The pod is removed from the Service's endpoints. Ingress controllers and other nodes learn this a little later, so some requests still arrive.
- At the same time, the
hook runs; the 10-second sleep above keeps the pod serving meanwhile. ThepreStop action is stable from Kubernetes 1.34, so the image needs no shell.sleep - The kubelet sends SIGTERM. Spring Boot's graceful shutdown (
, the default) stops accepting new requests and waits up toserver.shutdown=graceful (30 seconds by default) for the ones in progress.spring.lifecycle.timeout-per-shutdown-phase - If the pod is still running when
runs out, the kubelet sends SIGKILL.terminationGracePeriodSeconds
The grace period must cover the preStop
Common pitfalls
- A maximum the cluster cannot place: if the nodes are full, new pods stay
. The HPA counts them as replicas, the running pods keep the whole load, and nothing improves until the cluster autoscaler adds nodes, minutes before the JVM even starts. PlanPending with node capacity.maxReplicas - HPA and VPA on the same metric: a Vertical Pod Autoscaler that changes the CPU request moves the HPA's 100% mark. Use different signals, or run VPA with
, which only recommends. An in-place resize (beta in Kubernetes 1.33 and 1.34) does not change a running JVM's maximum heap or the thread counts it chose at startup.updateMode: "Off" - Metrics with no data: when Prometheus or the adapter is down, the HPA cannot compute that metric. The CPU safety net in
can still scale up, but the HPA skips any scale-down while a metric is missing. KEDA'shpa-requests.yaml instead sets a fixed replica count after a number of failed reads.fallback - Slow starts: every minute of startup and warm-up is a minute of extra pods. Spring Boot's CDS training run and the JDK's AOT cache (JEP 483) shorten startup; AOT method profiles (JEP 515) and
CRaC checkpoints also shorten the JIT warm-up the HPA sees. Neither JEP is in Java 21.
Which signal to use
| Workload | Signal | How | Watch out for |
|---|---|---|---|
| CPU-bound web service, no Prometheus | CPU utilization | ContainerResource | JIT warm-up after readiness; size the request for warm load |
| Web service with Prometheus | Requests per second per pod | Prometheus Adapter and a Podsprometheus | Exclude /actuator |
| Slow or blocking endpoints | In-flight requests per pod | The _gcountrate() | A slow dependency raises it too |
| Kafka consumers | Consumer group lag | KEDA kafka | Partition count caps replicas; rebalance on every change |
| Queue or batch workers | Backlog size | The broker's own metric through KEDA, or a gauge published outside the workers | Scale to zero needs a metric that exists while no worker runs, and an acceptable cold start |
| Memory-heavy service | Requests or CPU, not memory | Size the heap separately | The JVM returns heap slowly, so memory stays high after load drops |
Whichever signal you choose, warm new pods before they are ready, keep a scale-down window of several minutes, and let a removed pod finish its work; with CPU or queue lag, add a scale-up window that covers the warm-up. See also