Kubernetes: Liveness, Readiness & Startup Probes + Graceful Shutdown

Kubernetes probes determine whether a container is alive, ready to serve traffic, or still starting up. Misconfigured probes are one of the most common causes of production incidents: pods that restart in loops, or traffic routed to pods that aren't ready. Graceful shutdown ensures in-flight requests complete before a pod terminates. This article covers all three probe types, Spring Boot Actuator integration, and the full graceful shutdown sequence.

Share

Contents

Probe Types Overview
ProbeFailure ActionPurpose
LivenessRestart the containerDetect deadlocks or stuck processes that can only recover via restart
ReadinessRemove pod from Service endpoints (stop sending traffic)Detect when the app is temporarily unable to serve requests (e.g., DB connection lost)
StartupRestart the container (if still failing after deadline)Give slow-starting apps time to initialise without triggering liveness restarts

Each probe can use one of three mechanisms: httpGet, tcpSocket, or exec (command).

Spring Boot Actuator Health Endpoints

Spring Boot (2.3 and later) exposes separate /actuator/health/liveness and /actuator/health/readiness endpoints when it detects that it runs on Kubernetes. probes.enabled: true turns them on everywhere, so you can try them locally too. Each endpoint reports a group of health checks, and you choose what goes in each group:

YAML
# application.yml
management:
  endpoints:
    web:
      exposure:
        include: health,info,prometheus
  endpoint:
    health:
      probes:
        enabled: true        # enables /health/liveness and /health/readiness
      show-details: always   # optional — shows component details
      group:
        readiness:
          include: readinessState,db   # ready only when the database is reachable
        liveness:
          include: livenessState       # alive as long as the application itself is
Shell
# Verify endpoints
curl http://localhost:8080/actuator/health/liveness
# {"status":"UP"}

curl http://localhost:8080/actuator/health/readiness
# {"status":"UP","components":{"db":{"status":"UP"},"readinessState":{"status":"UP"}}}
Be careful about which dependencies you add to readiness. If a database or service that every pod uses goes down, every pod becomes not ready at once and Kubernetes stops sending traffic to all of them: clients get connection errors instead of a clear error response from your application. Add a dependency only if a pod without it really cannot do anything useful.

Liveness Probe

The liveness probe answers one question: is this process stuck so badly that only a restart helps? The settings below, in the container spec of a Deployment, check every 15 seconds and restart the container after three failures in a row:

YAML
livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  # Don't start checking until the app has had time to start
  initialDelaySeconds: 30
  # How often to check
  periodSeconds: 15
  # Timeout per check
  timeoutSeconds: 5
  # Consecutive successes to become healthy (must be 1 for liveness)
  successThreshold: 1
  # Consecutive failures before restarting the container
  failureThreshold: 3
  # Total liveness budget: 30 + (15 × 3) = 75 seconds before first restart
Liveness should only check that the process can respond, not database connectivity. A database outage should make the pod not ready, not restart it. Restarting won't fix a DB that's down, and cascading restarts will make recovery worse.

Readiness Probe

The readiness probe answers a different question: should this pod receive traffic right now? It is checked throughout the pod's life, not only at startup:

YAML
readinessProbe:
  httpGet:
    path: /actuator/health/readiness
    port: 8080
  initialDelaySeconds: 20    # less than liveness — check readiness sooner
  periodSeconds: 10
  timeoutSeconds: 5
  successThreshold: 1
  failureThreshold: 3        # remove from Service after 3 failures (30 seconds)

When a readiness probe fails, the pod is removed from the Service's endpoint list: no new requests are routed to it. The pod is not restarted. Once the dependency recovers, the probe passes and traffic resumes automatically.

Startup Probe (Slow-Start Apps)

Apps with long startup times (Spring Batch jobs, apps with many Flyway migrations) need a startup probe to prevent liveness from killing them before they finish initialising.

YAML
startupProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  # failureThreshold × periodSeconds = maximum startup time allowed
  failureThreshold: 30     # 30 attempts
  periodSeconds: 10        # = 300 seconds (5 minutes) before giving up
  timeoutSeconds: 5

# While startupProbe is running, liveness and readiness probes are DISABLED.
# Once startupProbe succeeds once, normal probes take over.
livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  periodSeconds: 15
  failureThreshold: 3

Custom Health Indicators

To report the health of something Spring Boot does not check by itself, implement HealthIndicator. Keep the check fast and cheap: it runs on every probe call.

Java
import org.springframework.boot.actuate.health.Health;
import org.springframework.boot.actuate.health.HealthIndicator;
import org.springframework.stereotype.Component;

@Component
public class PaymentGatewayHealthIndicator implements HealthIndicator {

    private final PaymentGatewayClient gateway;

    public PaymentGatewayHealthIndicator(PaymentGatewayClient gateway) {
        this.gateway = gateway;
    }

    @Override
    public Health health() {
        try {
            return gateway.ping()
                    ? Health.up().build()
                    : Health.down().withDetail("reason", "no response").build();
        } catch (Exception e) {
            return Health.down(e).build();
        }
    }
}

The indicator is named after its bean, without the HealthIndicator suffix: paymentGateway. It appears in /actuator/health straight away, but in the readiness probe only if you add it to that group: include: readinessState,paymentGateway. The warning in the previous section applies here too.

Waiting for warm-up before taking traffic

Spring Boot reports the application as ready only after all ApplicationRunner and CommandLineRunner beans have finished. Work done in a runner, such as filling a cache, therefore completes before the pod receives any traffic, with no extra code:

Java
import org.springframework.boot.ApplicationArguments;
import org.springframework.boot.ApplicationRunner;
import org.springframework.stereotype.Component;

@Component
public class CacheWarmup implements ApplicationRunner {

    private final ProductCache cache;

    public CacheWarmup(ProductCache cache) {
        this.cache = cache;
    }

    @Override
    public void run(ApplicationArguments args) {
        cache.loadAll();   // the readiness probe reports UP only after this returns
    }
}

To take a running pod out of traffic yourself, for example during a long maintenance task, publish a readiness change. Publish ACCEPTING_TRAFFIC the same way to bring it back:

Java
import org.springframework.boot.availability.AvailabilityChangeEvent;
import org.springframework.boot.availability.ReadinessState;
import org.springframework.context.ApplicationEventPublisher;
import org.springframework.stereotype.Component;

@Component
public class TrafficSwitch {

    private final ApplicationEventPublisher eventPublisher;

    public TrafficSwitch(ApplicationEventPublisher eventPublisher) {
        this.eventPublisher = eventPublisher;
    }

    public void stopTraffic() {
        AvailabilityChangeEvent.publish(eventPublisher, this, ReadinessState.REFUSING_TRAFFIC);
    }

    public void resumeTraffic() {
        AvailabilityChangeEvent.publish(eventPublisher, this, ReadinessState.ACCEPTING_TRAFFIC);
    }
}

Graceful Shutdown

When Kubernetes terminates a pod it sends SIGTERM. With graceful shutdown, Spring Boot then stops accepting new requests and waits for in-flight ones to complete before exiting. It is the default from Spring Boot 3.4; on older versions, turn it on yourself:

YAML
# application.yml — enable graceful shutdown
server:
  shutdown: graceful          # wait for in-flight requests to complete

spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s   # max wait time per phase
YAML
# Deployment spec — full graceful shutdown config
spec:
  template:
    spec:
      terminationGracePeriodSeconds: 60   # K8s waits this long before SIGKILL
      containers:
        - name: payment-service
          lifecycle:
            preStop:
              # Wait before SIGTERM, so every load balancer has stopped
              # sending new requests to this pod
              sleep:
                seconds: 10

The sleep action needs Kubernetes 1.30 or later. On older clusters, use exec with command: ["sh", "-c", "sleep 10"], which needs a shell in the image; distroless images have none.

The full shutdown sequence:

  1. The pod is marked Terminating. Two things start at the same time: Kubernetes begins removing the pod from the Service's endpoints, and the preStop hook runs.
  2. The preStop sleep gives that removal time to reach every node and load balancer. Without it, the pod could stop accepting requests while some are still being sent to it.
  3. SIGTERM sent to the container process: Spring Boot begins graceful shutdown.
  4. Spring Boot drains in-flight requests (up to timeout-per-shutdown-phase).
  5. If the process hasn't exited within terminationGracePeriodSeconds, Kubernetes sends SIGKILL.
Set terminationGracePeriodSeconds to at least preStop sleep + timeout-per-shutdown-phase + 10s buffer. A common mistake is setting the Spring Boot timeout longer than the K8s grace period: the JVM gets SIGKILL before it finishes draining.

Common Pitfalls
  • Liveness checks a dependency (DB, Redis): causes restart loops when a dependency is down. Keep liveness simple.
  • No startup probe for slow apps: liveness kills the app before it finishes Flyway migrations. Add a startup probe with failureThreshold × period ≥ max startup time.
  • initialDelaySeconds too short: liveness fires before Spring context is loaded, causing unnecessary restart on first boot.
  • No preStop sleep: requests are still routed to the pod for a few seconds after SIGTERM arrives (endpoint propagation delay). The preStop sleep prevents 5xx errors during rolling deploys.
  • terminationGracePeriodSeconds smaller than shutdown timeout: the JVM gets SIGKILL before draining completes. Always: terminationGracePeriod > preStop + Spring shutdown timeout.