Debezium PostgreSQL CDC with Kafka and Spring Boot: Production Guide

Debezium PostgreSQL change data capture reads committed row changes from the write-ahead log and publishes them to Kafka. It can keep a search index, cache, analytical store, or downstream database synchronized without adding a broker call to every application write path.

CDC is not “turn on a connector and forget it.” A production design must account for:

• the initial snapshot;
• PostgreSQL logical replication privileges and publications;
• replication-slot WAL retention;
• primary keys and replica identity;
• deletes, tombstones, and unchanged TOAST values;
• schema evolution;
• duplicate delivery and downstream idempotency;
• security of row-level data in Kafka.

Version note

This guide was reviewed against Debezium 3.6, Spring Boot 4.1, current Apache Kafka documentation, and PostgreSQL 18 documentation on August 10, 2026.

TL;DR

• Use PostgreSQL's built-in pgoutput logical decoding plug-in and a unique replication slot per connector.
• Have a DBA create a filtered publication and grant the connector only the required replication and table privileges.
• Treat the initial snapshot and streaming phase as one recovery plan.
• Require stable keys for captured tables and understand REPLICA IDENTITY before consuming updates and deletes.
• Monitor slot activity, connector offsets, lag, and retained WAL bytes. A stopped slot can fill the database disk.
• Make Spring Boot consumers idempotent; CDC delivery can repeat after restart or recovery.

Decide whether raw CDC or an outbox is the right contract

Debezium can capture application tables directly or capture an append-only outbox table. Those designs solve different problems.

Direct CDC exposes implementation details such as column names, normalization, and several row updates that may represent one business operation. It should not be relabeled as a domain-event stream without a deliberate transformation layer.

Use the transactional outbox pattern when downstream services need stable business semantics. Use direct CDC when consumers genuinely need the database change model.

Understand the PostgreSQL connector flow

On its first connection, Debezium normally takes a consistent snapshot of included tables because PostgreSQL no longer has every historical WAL record. It then streams from the exact logical position associated with that snapshot.

PostgreSQL tables
  -> initial snapshot (`op = r`)
  -> logical replication slot
  -> pgoutput change stream
  -> Debezium PostgreSQL connector
  -> Kafka topics
  -> Spring Boot projection consumer

For later changes, the operation field is commonly:

• c for create;
• u for update;
• d for delete;
• r for snapshot read.

The change envelope also contains before, after, source metadata, and connector timestamps. Exact serialization depends on the Kafka Connect converter, so consumers should be tested against the deployed converter and schema settings.

The default topic naming pattern is based on:

<topic.prefix>.<schema>.<table>

Treat topic.prefix as a stable namespace. Changing it creates new topic names and is not a harmless connector rename.

Configure PostgreSQL for logical decoding

The database must support logical replication. At the server level, review at least:

wal_level = logical
max_replication_slots = 10
max_wal_senders = 10

These are capacity examples, not universal values. Count every logical and physical replication consumer, include failover needs, and leave operational headroom.

Create a dedicated login rather than giving the connector the application's credentials:

create role debezium_orders
    with login replication password '<managed-secret>';

grant connect on database orders to debezium_orders;
grant usage on schema public to debezium_orders;
grant select on table public.customer, public.purchase_order
    to debezium_orders;

Have a database owner create the publication explicitly:

create publication orders_cdc_publication
    for table public.customer, public.purchase_order;

Manual publication management makes scope reviewable and avoids granting the connector broad table ownership or automatic-creation rights. When a new table is added, update both the publication and connector include list through the same deployment process.

PostgreSQL host-based authentication must also permit the connector host to establish the required normal and replication connections. Use TLS and a secret manager in production; never commit the password shown by a local example.

Create a narrow Debezium connector

A production-oriented baseline is:

{
  "name": "orders-postgres-cdc",
  "config": {
    "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
    "tasks.max": "1",
    "database.hostname": "orders-postgres",
    "database.port": "5432",
    "database.user": "debezium_orders",
    "database.password": "<injected-secret>",
    "database.dbname": "orders",
    "topic.prefix": "orders",
    "plugin.name": "pgoutput",
    "slot.name": "orders_cdc",
    "publication.name": "orders_cdc_publication",
    "publication.autocreate.mode": "disabled",
    "schema.include.list": "public",
    "table.include.list": "public.customer,public.purchase_order",
    "snapshot.mode": "initial",
    "heartbeat.interval.ms": "10000",
    "provide.transaction.metadata": "true"
  }
}

Important properties:

• plugin.name=pgoutput selects PostgreSQL's built-in logical output plug-in.
• slot.name identifies server-side replication progress and must be unique per active connector.
• publication.name must match the DBA-managed publication.
• include lists prevent accidental capture of unrelated personal or operational data.
• snapshot.mode=initial takes a snapshot only when no prior connector offset exists.
• heartbeat records make connector liveness and source position visible.
• transaction metadata is useful when consumers must observe source transaction boundaries; it does not make downstream writes atomic with PostgreSQL.

Connector configuration and Kafka Connect offsets are recovery-critical state. Deleting offsets, renaming the slot, or changing snapshot mode can cause a resnapshot, gaps, or duplicates depending on the remaining WAL. Treat those operations as migrations with a tested runbook.

Plan the initial snapshot

An initial snapshot can read every included row. Before starting it, estimate:

• total rows and bytes;
• read I/O and cache pressure;
• network and Kafka throughput;
• connector heap;
• schema locks needed at snapshot start;
• how long retained WAL must bridge snapshot completion;
• downstream capacity for the initial burst.

Snapshot records use read semantics (op=r). A projection consumer usually applies them as upserts, not as a separate business action. Sending an email or charging a customer for an r event would be a serious contract mistake.

For a large table added later, Debezium supports incremental snapshots. They read primary-key-ordered chunks while streaming continues and use a snapshot window to resolve collisions between snapshot rows and concurrent WAL changes. Test the mechanism with real update traffic and ensure the table has an efficient key for chunking.

Do not reset the connector merely to “try the snapshot again.” Record the reason, target tables, expected source position, and downstream reset procedure before any resnapshot.

Require a stable record key

For a table with a primary key, Debezium uses the key columns for the Kafka record key. That key is essential for:

• partitioning all changes for one row together;
• compacted-topic behavior;
• downstream upserts and deletes;
• per-key ordering.

A captured table without a primary key can produce records with null keys. That makes partitioning and delete handling much harder. Prefer adding a stable primary key.

PostgreSQL REPLICA IDENTITY controls which old row values are available for update and delete records:

• DEFAULT uses the primary key;
• USING INDEX selects a qualifying unique index;
• FULL records the old values of all columns;
• NOTHING records no old row identity.

REPLICA IDENTITY FULL increases WAL volume and is not a substitute for thoughtful key design. If a table cannot have a primary key, test the exact Debezium record key and delete envelope after configuring message.key.columns or another supported key strategy.

Handle Debezium envelopes in Spring Boot

For a JSON converter, a narrow consumer can inspect the operation and upsert or delete a projection:

@Component
public class CustomerProjectionListener {

    private final CustomerProjectionService projections;

    public CustomerProjectionListener(CustomerProjectionService projections) {
        this.projections = projections;
    }

    @KafkaListener(
        topics = "orders.public.customer",
        groupId = "customer-search-projection-v1"
    )
    public void on(ConsumerRecord<JsonNode, JsonNode> record) {
        JsonNode envelope = record.value();

        if (envelope == null) {
            // A compacted-topic tombstone; the delete change arrived separately.
            return;
        }

        String operation = envelope.path("op").asText();

        switch (operation) {
            case "c", "u", "r" -> projections.upsert(
                requiredKey(record.key()),
                envelope.path("after"),
                sourcePosition(envelope.path("source"))
            );
            case "d" -> projections.delete(
                requiredKey(record.key()),
                sourcePosition(envelope.path("source"))
            );
            default -> throw new UnsupportedCdcOperation(operation);
        }
    }
}

Keep the listener thin. The projection service should validate schema version, apply an idempotent or monotonic update, and commit its checkpoint with the projection change when writing to another database.

If a delete record is followed by a Kafka tombstone, do not confuse the two. The delete envelope carries source semantics; the null-value tombstone allows log-compacted Kafka topics to remove the key eventually.

Make projection writes idempotent

Debezium and Kafka Connect can resend records after a crash, rebalance, offset restoration, or operational replay. A downstream write must tolerate that.

For a simple “latest row” projection:

• use the source primary key as the target key;
• upsert c, u, and r operations;
• delete on d;
• store a source position or version;
• reject an older position after a newer one has been applied.

For non-convergent effects such as “increment balance,” do not interpret raw updates as commands. Store a processed change identity and apply the effect in one transaction, or transform the CDC record into a stable event contract first. The idempotent Kafka consumer guide shows the database pattern.

Kafka offset alone can checkpoint one consumer group, but it is not a portable business identity after records are copied or replayed to another topic. Preserve source metadata needed for diagnosis and ordering without putting high-cardinality positions into metric labels.

Do not turn unchanged TOAST data into null

PostgreSQL can store large column values out of line using TOAST. For some updates, unchanged TOASTed columns are not present in the WAL record. Debezium can emit a configured unavailable-value placeholder instead of the original value.

The consumer must distinguish:

• a real SQL NULL;
• an absent field due to converter or schema rules;
• Debezium's unavailable-value placeholder;
• a present new value.

Never overwrite a downstream column with null merely because an unchanged TOAST value was unavailable in an update event. Options include merging the update with existing projection state, using a complete-state query path, changing replica identity with full awareness of WAL cost, or designing an outbox payload that always contains the required fields.

Test this explicitly with large text, jsonb, and binary values.

Treat schema evolution as a coordinated deployment

PostgreSQL logical decoding does not emit DDL changes as normal data change events. Debezium refreshes table metadata as it encounters changes, but consumers do not receive a complete database migration plan.

Use an expand-and-contract sequence:

1. Add a nullable column or a column with a compatible default.
2. Deploy consumers that accept both old and new record shapes.
3. Deploy producers or database writers that populate the new field.
4. Backfill if necessary.
5. Enforce stronger constraints only after every reader is compatible.
6. Remove old fields in a later release after retention and replay windows pass.

Renaming a column is usually an incompatible remove-plus-add event for consumers. Changing numeric precision, timestamp semantics, enum representation, or a primary key deserves the same contract review as an API change.

Debezium's PostgreSQL connector does not support schema changes while an incremental snapshot is running. Coordinate migrations with snapshot operations instead of assuming they are independent.

Monitor replication slots and WAL retention

A logical replication slot tells PostgreSQL which WAL is still needed. If the connector is stopped or cannot advance, PostgreSQL retains that WAL. Disk usage can grow until the database becomes unavailable.

Inspect slot state:

select
    slot_name,
    active,
    restart_lsn,
    confirmed_flush_lsn,
    pg_size_pretty(
        pg_wal_lsn_diff(pg_current_wal_lsn(), restart_lsn)
    ) as retained_wal
from pg_replication_slots
where slot_type = 'logical';

Alert on:

• an expected slot becoming inactive;
• retained WAL bytes and growth rate;
• connector source lag;
• failed tasks and restart loops;
• Kafka Connect offset commit failures;
• snapshot duration and remaining tables;
• oldest unprocessed downstream record.

max_slot_wal_keep_size can cap how much WAL a slot retains, but exceeding the limit can make required WAL unavailable and invalidate recovery. It is a last-resort safety limit, not a replacement for alerting and a recovery plan.

Do not set the connector to drop its slot automatically on normal production stops. A dropped slot discards the server-side position and can force a new snapshot. Delete a slot only through an approved decommission or rebuild procedure after confirming no connector uses it.

Low traffic in captured tables can delay visible progress while other tables generate WAL. Debezium documents heartbeat options, including a source heartbeat action query for some PostgreSQL scenarios. Implement that only with a dedicated heartbeat table, correct publication scope, and measured write rate.

Plan for PostgreSQL failover

A connector slot and publication live on the PostgreSQL topology, not in the Spring Boot application. Before a failover, know:

• whether the PostgreSQL version and managed service support synchronized or failover logical slots;
• how the connector discovers the new primary;
• whether the slot position is preserved;
• which Kafka Connect offsets correspond to it;
• whether a resnapshot is required if WAL is missing;
• how downstream projections are reset without producing irreversible side effects.

Test planned and unplanned failover. A connector that restarts successfully is not proof that it resumed at the correct source position.

Secure the captured data

CDC can copy every included column into Kafka, where retention and access differ from the source database.

• Capture only required schemas, tables, and columns.
• Use connector column exclusion, masking, or transformation for secrets and personal data.
• Encrypt database and Kafka connections.
• Restrict Kafka topics and schema registry subjects with ACLs.
• Set retention according to data classification and deletion obligations.
• Prevent raw payloads from appearing in connector logs or dead-letter records.
• Audit who can change connector configuration.

Masking after sensitive data has already reached an unrestricted raw topic is too late.

Test the operational failures

Use Testcontainers or an equivalent real environment for the mechanics, then run a staging failure drill with the deployed Kafka Connect platform. Embedded substitutes cannot prove logical decoding or slot retention behavior.

Common mistakes

Treating every row change as a domain event

Database updates describe storage mutations. Use an outbox when consumers need business facts.

Capturing tables without stable keys

Null or mutable keys break partitioning, compaction, delete handling, and idempotent projections.

Watching connector status but not the slot

A failed connector can be quiet while PostgreSQL retains gigabytes of WAL. Monitor both systems.

Assuming DDL is replicated

PostgreSQL logical replication does not replicate schema and DDL commands. Coordinate migrations with consumers.

Resetting offsets casually

Kafka Connect offsets, snapshots, and replication-slot positions form one recovery story. Resetting one component can create gaps or a full replay.

Sending business side effects from snapshot records

op=r reconstructs current state. It is not a new customer action.

Production checklist

• Is direct table CDC the right contract, or should this be an outbox?
• Does each captured table have a stable primary key and appropriate replica identity?
• Are publication and include lists intentionally narrow?
• Is the replication user least-privileged and TLS-enabled?
• Are snapshot load and downstream burst capacity tested?
• Are connector offsets and slot recovery documented together?
• Are duplicates, deletes, tombstones, and TOAST placeholders handled?
• Is schema evolution additive and coordinated?
• Are slot activity, retained WAL, connector lag, and consumer lag alerted?
• Has PostgreSQL failover and downstream replay been rehearsed?

CDC becomes reliable when it is operated as a data pipeline, not treated as an application annotation. The connector, replication slot, Kafka topics, schema contract, and idempotent consumers are all parts of the same system.

Official references

• Debezium 3.6: PostgreSQL connector
• Debezium: Incremental snapshots
• PostgreSQL: Logical replication configuration
• PostgreSQL: Publications
• PostgreSQL: Replication slots
• PostgreSQL: Replica identity
• PostgreSQL: Logical replication restrictions
• Apache Kafka documentation