Event-Driven Cache Invalidation with Spring Boot, Kafka, and Redis

A distributed cache reduces read latency and database load, but it introduces a second representation of the data. Once PostgreSQL changes, Redis may still contain the previous value. The hard part is not calling cache.evict(). The hard part is preserving a clear consistency model across database commits, Kafka delivery, Redis failures, concurrent readers, and several service deployments.

Event-driven invalidation is useful when a change in one bounded context affects caches owned by other services. It is not strong consistency and it does not make Redis part of the PostgreSQL transaction.

A reliable design should explain:

how a committed database change produces a durable event;
which Kafka consumer groups must receive that event;
what happens when events are duplicated or reordered;
how a cache miss racing with an update avoids restoring stale data;
how TTL limits the damage when messaging or invalidation fails;
whether all application instances share one Redis cache or own local caches;
how cache stampedes are controlled after a popular key is invalidated.

This guide uses Spring Boot 4.1, Java 25, PostgreSQL, Apache Kafka, and Redis.

TL;DR Keep PostgreSQL authoritative. Write a domain-change event to an outbox in the same transaction as the business update. Give each independent cache projection its own Kafka consumer group. Treat invalidation as at-least-once and idempotent. Use TTL as a fallback, and use a version watermark when a stale cache refill would be unacceptable.

Define the Consistency Contract

A cache design needs a specific contract rather than the phrase “always fresh.”

Examples:

Product descriptions may be stale for at most five minutes.
Inventory availability is never decided from Redis.
A price cache is invalidated within the Kafka-lag objective.
Payment and authorization decisions always read the source of truth.

For each field, decide:

Data	Cacheable?	Acceptable staleness	Authoritative check
Product description	Yes	Minutes	PostgreSQL
Display price	Often	Seconds	Pricing service
Inventory count	Carefully	Very short	Inventory transaction
Account balance	Usually no	None	Ledger database
Feature metadata	Yes	Minutes	Configuration store

Event-driven invalidation narrows a stale-data window. It does not remove the window between the database commit and event consumption.

Choose the Correct Topology

One shared Redis cache

All replicas of one service use the same Redis keyspace.

Service replicas
    \  |  /
     Redis

One Kafka consumer group for that cache projection is enough. Kafka delivers each partition record to one consumer in the group, and that consumer deletes the shared Redis key. Every replica observes the same deletion.

Separate Redis caches by service

ProductChanged topic
    |-- group catalog-cache-v1 -> catalog Redis
    |-- group search-cache-v1  -> search Redis
    +-- group rec-cache-v1     -> recommendation Redis

Each independent projection needs its own group ID. Reusing one group across those services load-balances events between them; it does not broadcast every event to every service.

In-process L1 caches

When every application instance has a private Caffeine or in-memory cache, one shared Kafka group does not notify every instance. Each event goes to only one group member.

Safer options include:

use Redis as the shared cache and avoid a private L1;
give L1 entries a very short TTL;
use a dedicated fan-out mechanism for local-cache eviction;
make L1 an optimization that can tolerate missed invalidations.

The topology must be decided before choosing a group ID.

Dependencies for Spring Boot 4.1

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>4.1.0</version>
    <relativePath />
</parent>

<properties>
    <java.version>25</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-webmvc</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-jpa</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-data-redis</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-cache</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-kafka</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
    <dependency>
        <groupId>org.postgresql</groupId>
        <artifactId>postgresql</artifactId>
        <scope>runtime</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

Let Spring Boot manage compatible Kafka, Redis, and Jackson versions. Do not use a snapshot parent or override Spring Kafka without a tested reason.

Application Configuration

spring:
  application:
    name: catalog-service

  datasource:
    url: ${DATABASE_URL}
    username: ${DATABASE_USERNAME}
    password: ${DATABASE_PASSWORD}

  jpa:
    hibernate:
      ddl-auto: validate

  data:
    redis:
      host: ${REDIS_HOST:localhost}
      port: ${REDIS_PORT:6379}
      connect-timeout: 2s
      timeout: 2s

  kafka:
    bootstrap-servers: ${KAFKA_BOOTSTRAP_SERVERS:localhost:9092}
    consumer:
      group-id: catalog-product-cache-v1
      enable-auto-commit: false
    listener:
      ack-mode: record

cache:
  product:
    ttl: 10m
    watermark-ttl: 24h

Use Flyway or Liquibase for schema changes. Do not configure spring.json.trusted.packages: "*". Use an explicit event type, exact package, or a schema-managed format.

Model the Cache Value Explicitly

Do not cache a JPA entity with lazy associations and persistence behavior. Map it to an immutable cache DTO.

public record ProductCacheValue(
        long id,
        String name,
        String description,
        BigDecimal price,
        long version
) {
    public static ProductCacheValue from(Product product) {
        return new ProductCacheValue(
                product.getId(),
                product.getName(),
                product.getDescription(),
                product.getPrice(),
                product.getVersion()
        );
    }
}

The version is part of the cache contract.

@Entity
@Table(name = "products")
public class Product {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    @Column(nullable = false)
    private String name;

    private String description;

    @Column(nullable = false, precision = 19, scale = 2)
    private BigDecimal price;

    @Version
    private long version;

    protected Product() {}

    public void update(
            String name,
            String description,
            BigDecimal price
    ) {
        this.name = name;
        this.description = description;
        this.price = price;
    }

    // Getters omitted.
}

Use BigDecimal for monetary values rather than double.

Publish a Domain Change, Not a Cache Command

An event should describe what happened:

public record ProductChanged(
        UUID eventId,
        long productId,
        long version,
        ChangeType changeType,
        Instant occurredAt
) {}

public enum ChangeType {
    CREATED,
    UPDATED,
    DELETED
}

Avoid a generic event containing an arbitrary cacheName and Object key. That couples producers to every consumer's cache implementation and lets malformed events target unrelated caches.

Each consumer decides which of its own cache keys are affected by a ProductChanged event.

Persist the Event with the Database Change

@TransactionalEventListener(AFTER_COMMIT) prevents publication after a rollback, but it does not make Kafka publication durable. The process can crash after PostgreSQL commits and before Kafka acknowledges the message.

Use a transactional outbox.

CREATE TABLE cache_change_outbox (
    id UUID PRIMARY KEY,
    aggregate_type VARCHAR(100) NOT NULL,
    aggregate_id VARCHAR(200) NOT NULL,
    event_type VARCHAR(150) NOT NULL,
    aggregate_version BIGINT NOT NULL,
    payload JSONB NOT NULL,
    status VARCHAR(30) NOT NULL DEFAULT 'READY',
    attempts INTEGER NOT NULL DEFAULT 0,
    next_attempt_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    published_at TIMESTAMPTZ
);

CREATE INDEX idx_cache_change_outbox_ready
    ON cache_change_outbox (
        next_attempt_at,
        created_at
    )
    WHERE status = 'READY';

The product update and outbox insert share one PostgreSQL transaction.

@Service
public class ProductCommandService {

    private final ProductRepository products;
    private final CacheChangeOutboxRepository outbox;
    private final EntityManager entityManager;
    private final ObjectMapper objectMapper;

    public ProductCommandService(
            ProductRepository products,
            CacheChangeOutboxRepository outbox,
            EntityManager entityManager,
            ObjectMapper objectMapper
    ) {
        this.products = products;
        this.outbox = outbox;
        this.entityManager = entityManager;
        this.objectMapper = objectMapper;
    }

    @Transactional
    public ProductResult update(
            long productId,
            UpdateProductCommand command
    ) {
        Product product = products
                .findById(productId)
                .orElseThrow();

        product.update(
                command.name(),
                command.description(),
                command.price()
        );

        entityManager.flush();

        ProductChanged event = new ProductChanged(
                UUID.randomUUID(),
                product.getId(),
                product.getVersion(),
                ChangeType.UPDATED,
                Instant.now()
        );

        outbox.save(
                CacheChangeOutbox.from(
                        event,
                        objectMapper.valueToTree(event)
                )
        );

        return ProductResult.from(product);
    }
}

Flushing obtains the updated optimistic-lock version before the outbox payload is created. The flush does not commit independently; both records still roll back together if the transaction fails.

For a hard delete, capture a tombstone version before removing the row. Do not reuse deleted product IDs.

Outbox Relay Semantics

A relay:

claims a bounded batch;
publishes each event to Kafka;
waits for the Kafka send result;
marks the outbox row published;
retries failures with backoff.

Use the product ID as the Kafka key:

ProducerRecord<String, ProductChanged> record =
        new ProducerRecord<>(
                "product.changed.v1",
                Long.toString(event.productId()),
                event
        );

kafkaTemplate.send(record)
        .get(10, TimeUnit.SECONDS);

Using one key keeps events for one product in one partition when all producers use the same topic and partitioning rule.

The relay is still at-least-once. It can crash after Kafka accepts the event and before PostgreSQL records published_at. Consumers must tolerate duplicates.

Basic Invalidation Is Idempotent

For a shared Redis cache, repeated deletion is harmless.

@Component
public class ProductCacheInvalidationListener {

    private final ProductCacheRepository cache;

    public ProductCacheInvalidationListener(
            ProductCacheRepository cache
    ) {
        this.cache = cache;
    }

    @KafkaListener(
            topics = "product.changed.v1",
            groupId = "catalog-product-cache-v1"
    )
    public void onProductChanged(
            ProductChanged event
    ) {
        cache.invalidate(
                event.productId(),
                event.version()
        );
    }
}

Do not catch every exception, log it, and return. Returning normally allows the listener container to commit the offset even though Redis invalidation failed.

Let the configured Kafka error handler retry. After the retry budget, route the event to a dead-letter topic and alert. TTL remains the final staleness bound.

The Stale Refill Race

A plain cache-aside implementation still has a race:

Reader misses Redis
Reader loads product version 10 from PostgreSQL
Writer commits product version 11
Invalidation event deletes the Redis key
Reader stores version 10 after the deletion

The stale value can remain until TTL even though the invalidation event was processed successfully.

Deleting twice after a delay can reduce this risk, but it is a timing heuristic rather than a proof.

For data where this race matters, keep a version watermark in Redis.

Version Watermark Design

Use two Redis keys:

product:data:{id}       -> serialized ProductCacheValue
product:watermark:{id}  -> highest committed version observed

When an invalidation event arrives:

update the watermark to the maximum of the existing and incoming version;
delete the cached value;
perform both operations atomically.

When a reader loads a value from PostgreSQL:

compare the database version with the watermark;
store it only when it is not older;
perform the comparison and write atomically.

This closes both possible interleavings:

stale refill first, then invalidation: invalidation deletes it;
invalidation first, then stale refill: the watermark rejects it.

Atomic Invalidation Script

invalidate-product.lua:

local incoming = tonumber(ARGV[1])
local watermarkTtl = tonumber(ARGV[2])
local current = tonumber(redis.call('GET', KEYS[2]) or '-1')

if incoming > current then
    redis.call(
        'SET',
        KEYS[2],
        tostring(incoming),
        'PX',
        watermarkTtl
    )
end

redis.call('DEL', KEYS[1])

return 1

An older out-of-order event may cause one unnecessary cache miss, but it cannot lower the watermark.

Atomic Versioned Put Script

put-product-if-current.lua:

local candidate = tonumber(ARGV[1])
local payload = ARGV[2]
local dataTtl = tonumber(ARGV[3])
local watermark = tonumber(
    redis.call('GET', KEYS[2]) or '-1'
)

if candidate < watermark then
    return 0
end

redis.call(
    'SET',
    KEYS[1],
    payload,
    'PX',
    dataTtl
)

return 1

Redis executes each Lua script atomically. An invalidation cannot interleave between the version comparison and the cache write.

Redis Repository

Using StringRedisTemplate keeps keys, versions, and JSON explicit.

@Component
public class ProductCacheRepository {

    private static final Duration DATA_TTL =
            Duration.ofMinutes(10);

    private static final Duration WATERMARK_TTL =
            Duration.ofHours(24);

    private final StringRedisTemplate redis;
    private final ObjectMapper objectMapper;
    private final RedisScript<Long> invalidateScript;
    private final RedisScript<Long> putScript;

    public ProductCacheRepository(
            StringRedisTemplate redis,
            ObjectMapper objectMapper,
            @Qualifier("invalidateProductScript")
            RedisScript<Long> invalidateScript,
            @Qualifier("putProductScript")
            RedisScript<Long> putScript
    ) {
        this.redis = redis;
        this.objectMapper = objectMapper;
        this.invalidateScript = invalidateScript;
        this.putScript = putScript;
    }

    public Optional<ProductCacheValue> get(
            long productId
    ) {
        String payload = redis.opsForValue()
                .get(dataKey(productId));

        if (payload == null) {
            return Optional.empty();
        }

        try {
            return Optional.of(
                    objectMapper.readValue(
                            payload,
                            ProductCacheValue.class
                    )
            );
        } catch (JsonProcessingException exception) {
            redis.delete(dataKey(productId));
            throw new CacheSerializationException(
                    productId,
                    exception
            );
        }
    }

    public boolean putIfCurrent(
            ProductCacheValue value
    ) {
        String payload;

        try {
            payload = objectMapper
                    .writeValueAsString(value);
        } catch (JsonProcessingException exception) {
            throw new CacheSerializationException(
                    value.id(),
                    exception
            );
        }

        Long result = redis.execute(
                putScript,
                List.of(
                        dataKey(value.id()),
                        watermarkKey(value.id())
                ),
                Long.toString(value.version()),
                payload,
                Long.toString(DATA_TTL.toMillis())
        );

        return Long.valueOf(1L).equals(result);
    }

    public void invalidate(
            long productId,
            long version
    ) {
        redis.execute(
                invalidateScript,
                List.of(
                        dataKey(productId),
                        watermarkKey(productId)
                ),
                Long.toString(version),
                Long.toString(
                        WATERMARK_TTL.toMillis()
                )
        );
    }

    private String dataKey(long productId) {
        return "product:data:" + productId;
    }

    private String watermarkKey(long productId) {
        return "product:watermark:" + productId;
    }
}

Script beans:

@Configuration
public class ProductCacheScripts {

    @Bean
    RedisScript<Long> invalidateProductScript() {
        return RedisScript.of(
                new ClassPathResource(
                        "redis/invalidate-product.lua"
                ),
                Long.class
        );
    }

    @Bean
    RedisScript<Long> putProductScript() {
        return RedisScript.of(
                new ClassPathResource(
                        "redis/put-product-if-current.lua"
                ),
                Long.class
        );
    }
}

Spring Data Redis uses Redis script caching and attempts EVALSHA before falling back to EVAL.

The watermark TTL must exceed the maximum realistic lifetime of a concurrent stale read and the data-cache TTL. Keeping it forever may be appropriate for a bounded entity set, but an unbounded keyspace needs a retention policy.

Cache-Aside Read Path

@Service
public class ProductQueryService {

    private final ProductRepository products;
    private final ProductCacheRepository cache;

    public ProductQueryService(
            ProductRepository products,
            ProductCacheRepository cache
    ) {
        this.products = products;
        this.cache = cache;
    }

    @Transactional(readOnly = true)
    public ProductView findById(long productId) {
        Optional<ProductCacheValue> cached =
                cache.get(productId);

        if (cached.isPresent()) {
            return ProductView.from(
                    cached.get()
            );
        }

        Product product = products
                .findById(productId)
                .orElseThrow();

        ProductCacheValue candidate =
                ProductCacheValue.from(product);

        cache.putIfCurrent(candidate);

        return ProductView.from(candidate);
    }
}

If putIfCurrent rejects the value because a newer event already advanced the watermark, the current request may still hold an older database snapshot. For a user-facing response that must not return the stale value, end the transaction and retry the database read once from a new transaction.

Most catalog use cases can return the value loaded by the current transaction while preventing it from polluting Redis. Stronger domains should bypass the cache and use an authoritative read.

Why `@Cacheable` Alone Is Not Enough Here

Spring Cache is useful for ordinary cache-aside logic:

@Cacheable(
    cacheNames = "products",
    key = "#productId"
)
public ProductView findById(long productId) {
    return loadFromDatabase(productId);
}

However, the version-watermark design needs an atomic compare-and-set that the generic cache abstraction does not express.

Also be careful when combining @Transactional and @CacheEvict. Cache and transaction advice are separate interceptors. A method can evict after returning while the database transaction is still completing through its proxy chain.

For durable cross-service invalidation:

write the outbox inside the transaction;
let the Kafka-driven invalidator update Redis;
optionally perform an after-commit local eviction as a latency optimization;
do not rely on the local callback as the only delivery path.

TTL Is a Safety Net

Every cache entry should expire even when invalidation is event-driven.

TTL protects against:

a Kafka event that never reaches the consumer;
a DLT event awaiting repair;
a configuration mistake in one deployment;
an incorrect cache key;
a Redis restore containing old values;
an operator pausing the invalidation consumer.

Choose TTL from the business staleness budget, not from a desired hit ratio alone.

Add jitter to large populations of keys so they do not expire simultaneously:

public Duration productTtl() {
    long baseSeconds =
            Duration.ofMinutes(10).toSeconds();

    long jitter =
            ThreadLocalRandom.current()
                    .nextLong(0, 60);

    return Duration.ofSeconds(
            baseSeconds + jitter
    );
}

A longer TTL improves hit ratio but increases the maximum stale period when event delivery fails.

Collection Caches Need Different Keys

Caching a full allProducts list and clearing the entire products cache after every change is expensive and can evict unrelated hot entries.

For query or collection caches, use a generation number:

product-list:generation -> 42
product-list:42:category:books:page:0

When a product changes, increment the generation. New reads use generation 43; old list keys expire naturally.

This avoids scanning or deleting thousands of query keys.

return redis.call(
    'INCR',
    'product-list:generation'
)

Entity-key invalidation and query-cache invalidation are separate concerns. A product update may affect category lists, search results, price ranges, and recommendation projections.

Cache Stampede After Invalidation

A popular key can receive thousands of concurrent misses immediately after deletion.

Mitigations include:

request coalescing inside one instance;
a short, bounded per-key distributed lock;
soft TTL plus stale-while-revalidate;
asynchronous refresh;
TTL jitter;
provider or database rate limits.

Do not create an unbounded virtual thread for every cache miss and assume the database will absorb it.

A lock is only a load-control optimization. PostgreSQL remains the source of truth, and the read path must still work when the lock service is unavailable according to the chosen failure policy.

Kafka Error Handling

A cache deletion is idempotent, so record redelivery is safe.

@Configuration
public class CacheInvalidationErrorConfiguration {

    @Bean
    DefaultErrorHandler cacheErrorHandler(
            KafkaTemplate<Object, Object> template
    ) {
        DeadLetterPublishingRecoverer recoverer =
                new DeadLetterPublishingRecoverer(
                        template,
                        (record, exception) ->
                                new TopicPartition(
                                        record.topic() + ".DLT",
                                        record.partition()
                                )
                );

        return new DefaultErrorHandler(
                recoverer,
                new ExponentialBackOffWithMaxRetries(5)
        );
    }
}

Do not swallow Redis connection errors in the listener. If processing fails, the container needs to see the exception so it can retry.

A DLT is not completion. Alert on it and provide an operator procedure to replay the event after the Redis or configuration problem is fixed.

Event Ordering and Versions

Kafka preserves order within one partition, not across the entire topic.

Use the aggregate ID as the key, but still keep the version because:

producers may use inconsistent keys;
topics can be replayed;
a consumer can receive duplicates;
multiple topics may influence one cache;
an operator may republish old events;
the Redis write race exists independently of Kafka order.

The watermark stores the greatest observed version, so an older event cannot make an older cache value valid again.

For independent services that derive different versions, do not compare unrelated counters. Use a revision owned by the authoritative producer or a globally ordered change token.

Redis Failure Behavior

Decide whether cache failure is fail-open or fail-closed.

For ordinary catalog reads, fail-open normally means:

Redis unavailable
-> load from PostgreSQL
-> return the result
-> do not fail the user request only because caching failed

For authentication, quotas, or security state, blindly failing open may be unsafe. Those are not ordinary cache use cases.

Do not restore an old Redis snapshot and assume its values are fresh. Flush affected cache namespaces or preserve version watermarks carefully before serving restored entries.

Redis persistence is optional for a rebuildable cache. Enabling AOF does not turn Redis into the source of truth.

Serialization and Schema Evolution

The original example used GenericJackson2JsonRedisSerializer. Spring Data Redis 4 provides Jackson 3 serializers such as JacksonJsonRedisSerializer and GenericJacksonJsonRedisSerializer; the older Jackson 2 variants are deprecated.

For one cache type, a typed serializer is safer than unrestricted polymorphic typing:

RedisSerializer<ProductCacheValue> serializer =
        new JacksonJsonRedisSerializer<>(
                ProductCacheValue.class
        );

When several application versions read the same Redis values:

add fields compatibly;
tolerate missing optional fields;
use a cache-key schema version;
invalidate the old namespace during incompatible changes;
avoid serializing implementation-specific JPA proxy types.

Example namespace:

catalog:v2:product:data:123

Changing the prefix is often safer than trying to deserialize every historical cache shape.

Security

Cache and invalidation infrastructure can expose sensitive data.

Do not cache secrets by default

Avoid caching:

access tokens;
passwords;
payment-card data;
unredacted personal profiles;
authorization decisions without a strict expiry model;
entire HTTP request or response bodies.

Protect Redis

Use:

private networking;
authentication;
TLS where required;
least-privilege ACLs;
separate key prefixes or databases by environment;
memory limits and eviction policy monitoring.

Validate events

Reject malformed events before constructing Redis keys.

public void validate(ProductChanged event) {
    if (event.productId() <= 0) {
        throw new IllegalArgumentException(
                "Invalid product ID"
        );
    }

    if (event.version() < 0) {
        throw new IllegalArgumentException(
                "Invalid product version"
        );
    }
}

Do not allow the event payload to select an arbitrary Redis command, key prefix, or cache name.

Observability

Track both performance and freshness.

Cache metrics

hit and miss count;
hit ratio;
cache load duration;
Redis command latency;
Redis errors;
cached-value size;
eviction count;
rejected stale put count;
watermark update count;
stampede-lock contention.

Spring Boot Actuator can instrument supported cache implementations with metrics prefixed by cache.

Kafka metrics

consumer lag;
event age at consumption;
retry count;
DLT count;
deserialization failures;
rebalance count;
outbox backlog and oldest-row age.

Freshness metrics

A high hit ratio can hide stale data. Also measure:

database commit time
event occurred time
Kafka consumption time
Redis invalidation completion time

Useful derived values:

outbox delay
Kafka transport delay
consumer processing delay
total invalidation delay

Do not use product IDs as metric labels. Use bounded labels such as service, cache, event type, and outcome.

Testing the Failure Windows

Use PostgreSQL, Kafka, and Redis Testcontainers for integration tests.

Database rollback

fail the product update;
verify no outbox row exists;
fail the outbox insert;
verify the product update rolls back.

Duplicate event

deliver the same eventId and version twice;
verify the operation remains safe;
verify the watermark does not move backward.

Out-of-order event

process version 12;
then process version 11;
verify watermark 12 remains;
verify version 11 cannot become cached.

Stale refill race

Coordinate these steps:

reader loads version 10 from PostgreSQL;
writer commits version 11;
consumer invalidates with watermark 11;
reader attempts to cache version 10;
verify the Lua script rejects the put.

Also reverse steps 3 and 4 and verify the later invalidation deletes version 10.

Kafka publication crash

publish an outbox event successfully;
stop the relay before marking it published;
restart;
verify the duplicate event is harmless.

Redis outage

make Redis unavailable during invalidation;
verify Kafka retry occurs;
verify the offset is not committed as success;
verify reads follow the chosen database fallback policy.

DLT recovery

send a malformed or repeatedly failing event;
verify it reaches the DLT;
repair the cause;
replay it and verify the cache converges.

Consumer-group topology

Start two different cache projections:

catalog-product-cache-v1
search-product-cache-v1

Verify both receive the same product event. Then start several replicas in one group and verify only one replica handles each partition record while all share the same Redis result.

Troubleshooting

The database changed, but Redis still has old data

Check:

outbox row creation;
relay backlog;
Kafka send result;
topic name and event key;
consumer group ID;
consumer lag;
Redis connection errors;
data and watermark key prefixes;
DLT records;
TTL.

Only one service invalidates its cache

Check whether several independent services accidentally share the same Kafka group ID. Kafka load-balances records within one group.

Every service instance must receive the event, but only one does

The instances probably have private local caches while sharing one consumer group. Use shared Redis, a true fan-out mechanism, or short-lived L1 entries.

Stale data returns after a successful invalidation

Check for the stale refill race. A cache miss may have loaded an old database version before the commit and stored it after eviction. Use the version-watermark scripts or a design with equivalent conditional writes.

Kafka events are occasionally missing

Check whether publication relies only on @TransactionalEventListener(AFTER_COMMIT). That callback is not a durable outbox. Persist the event in the database transaction and relay it.

The consumer logs an error but Kafka does not retry

Check for broad catch blocks that log and return. Let the listener throw so the container error handler sees the failure.

Redis memory grows continuously