Spring Kafka: Error Handling & Retry

Spring Kafka provides several error handling strategies, from simple logging to automatic retry with backoff and dead letter topics, so that transient failures don't cause message loss.

Default Error Handler
Fixed & Exponential Backoff Retry
@RetryableTopic
Dead Letter Topic
Wiring the Error Handler

Default Error Handler

Since Spring Kafka 2.8, DefaultErrorHandler is the default. When a listener throws an exception, it logs the error and, by default, seeks back to the failed offset so the record is retried immediately up to 10 times before being skipped.

Java

// This is wired automatically — you don't need to declare it unless customising
import org.springframework.kafka.listener.DefaultErrorHandler;

// The legacy SeekToCurrentErrorHandler (pre-2.8) behaved similarly
// but DefaultErrorHandler supersedes it and supports BackOff natively

The default 10-retry limit with no delay means all retries happen back-to-back in milliseconds. For transient failures (network timeouts, DB unavailability) you almost always want to add a backoff policy.

Fixed & Exponential Backoff Retry

Configure FixedBackOff for a constant delay between attempts, or ExponentialBackOff to double the wait time after each failure.

Java

import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.util.backoff.ExponentialBackOff;
import org.springframework.util.backoff.FixedBackOff;

// Fixed: retry up to 3 times with a 2-second pause between attempts
DefaultErrorHandler fixedHandler = new DefaultErrorHandler(
    new FixedBackOff(2000L, 3L) // interval ms, maxAttempts
);

// Exponential: start at 1s, double each time, cap at 30s, up to 5 attempts
ExponentialBackOff expBackOff = new ExponentialBackOff(1000L, 2.0);
expBackOff.setMaxInterval(30_000L);
expBackOff.setMaxElapsedTime(120_000L); // stop after 2 minutes total
DefaultErrorHandler expHandler = new DefaultErrorHandler(expBackOff);

During blocking retries, the consumer thread is paused and does not poll Kafka. If the retry window exceeds max.poll.interval.ms, the broker will kick the consumer out of its group, triggering a rebalance. Keep total retry time under that threshold or use @RetryableTopic for non-blocking retry.

@RetryableTopic

@RetryableTopic implements non-blocking retry: failed messages are published to dedicated retry topics (orders-retry-1, orders-retry-2, …) and consumed after the configured delay. The main consumer continues processing other messages immediately.

Java

import org.springframework.kafka.annotation.DltHandler;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.annotation.RetryableTopic;
import org.springframework.kafka.retrytopic.TopicSuffixingStrategy;
import org.springframework.retry.annotation.Backoff;
import org.springframework.stereotype.Service;

@Service
public class OrderConsumer {

    @RetryableTopic(
        attempts = "4",                        // 1 original + 3 retries
        backoff = @Backoff(delay = 1000, multiplier = 2.0, maxDelay = 10000),
        autoCreateTopics = "true",             // auto-create retry + DLT topics
        topicSuffixingStrategy = TopicSuffixingStrategy.SUFFIX_WITH_DELAY_VALUE
    )
    @KafkaListener(topics = "orders")
    public void consume(String message) {
        System.out.println("Processing: " + message);
        if (message.contains("BAD")) {
            throw new RuntimeException("Simulated processing failure");
        }
    }

    @DltHandler
    public void handleDlt(String message) {
        System.err.println("DLT received: " + message);
        // save to error store, alert, or re-queue manually
    }
}

Enable @RetryableTopic by adding @EnableKafkaRetryTopic to a @Configuration class (or enable it via @EnableKafka in Spring Kafka 3.x where it is enabled automatically).

Dead Letter Topic

After all retry attempts are exhausted, messages that still fail are routed to a Dead Letter Topic (DLT). Annotate a method with @DltHandler to process DLT messages, or configure a DeadLetterPublishingRecoverer manually.

Java

import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.listener.DeadLetterPublishingRecoverer;
import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.util.backoff.FixedBackOff;

@Configuration
public class KafkaErrorConfig {

    // Manual DLT configuration (alternative to @RetryableTopic)
    @Bean
    public DefaultErrorHandler errorHandler(KafkaTemplate<Object, Object> template) {
        // By default, publishes to <original-topic>.DLT
        DeadLetterPublishingRecoverer recoverer = new DeadLetterPublishingRecoverer(template);

        return new DefaultErrorHandler(recoverer, new FixedBackOff(1000L, 3L));
    }
}

Java

// @DltHandler — used with @RetryableTopic for in-class DLT handling
import org.springframework.kafka.annotation.DltHandler;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.messaging.handler.annotation.Header;

@DltHandler
public void processDlt(
        String message,
        @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {
    System.err.printf("Failed message from topic %s: %s%n", topic, message);
    // persist to database, send alert, etc.
}

Wiring the Error Handler

Register the DefaultErrorHandler on the ConcurrentKafkaListenerContainerFactory. You can also mark certain exceptions as non-retryable so they go straight to the DLT.

Java

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.listener.DeadLetterPublishingRecoverer;
import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.util.backoff.ExponentialBackOff;

@Configuration
public class KafkaConsumerConfig {

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, String> kafkaListenerContainerFactory(
            ConsumerFactory<String, String> consumerFactory,
            KafkaTemplate<Object, Object> kafkaTemplate) {

        ExponentialBackOff backOff = new ExponentialBackOff(1000L, 2.0);
        backOff.setMaxAttempts(4);

        DeadLetterPublishingRecoverer recoverer = new DeadLetterPublishingRecoverer(kafkaTemplate);
        DefaultErrorHandler errorHandler = new DefaultErrorHandler(recoverer, backOff);

        // Skip retry entirely for these — send straight to DLT
        errorHandler.addNotRetryableExceptions(
            IllegalArgumentException.class,
            NullPointerException.class
        );

        ConcurrentKafkaListenerContainerFactory<String, String> factory =
            new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(consumerFactory);
        factory.setCommonErrorHandler(errorHandler);

        return factory;
    }
}

Use addNotRetryableExceptions() for deserialization errors and validation failures that will never succeed on retry. Always add DeserializationException.class to the non-retryable list to avoid infinite retry loops on malformed messages.

Contents

Default Error Handler

Fixed & Exponential Backoff Retry

@RetryableTopic

Dead Letter Topic

Wiring the Error Handler

Related articles