When RabbitMQ Round-Robin Broke WebSocket Delivery Across Multiple Servers

A message broker can deliver a message successfully while the user still receives nothing. I ran into that distinction while scaling a real-time WebSocket application from one server to several application servers. The first routing improvement looked correct in a single-server test: instead of sending every event to every connected client, each client subscribed to a destination based on its user ID. Then I added another WebSocket server. Some messages reached the broker, entered the queue, and were consumed without an error—but never reached the intended browser. The failure was not random message loss inside RabbitMQ. The queue was doing exactly what it was configured to do. The topology no longer matched where WebSocket sessions lived. This post explains the failure, why the first fix was incomplete, and how I separated broker routing from WebSocket session ownership.

TL;DR

• Multiple consumers on one RabbitMQ queue are competing consumers, not broadcast recipients.
• RabbitMQ normally distributes deliveries among active consumers.
• A target user's WebSocket session exists on one application server unless session state is shared or messages are fanned out.
• If a shared queue delivers the event to another server, that server can consume the message successfully but cannot deliver it to the target session.
• I fixed the mismatch by giving each WebSocket server its own queue, binding those queues through a tenant-scoped exchange, and retaining the user ID as the application routing key.
• Multi-server tests were essential; the single-server test could not expose this topology error.

The Original System Assumed One Server

The system delivered real-time events to connected users through WebSocket sessions. Its business functions are not relevant to the routing problem, so the examples in this article use generic users and events. The original design assumed:

• one tenant boundary;
• one WebSocket application server;
• one shared messaging path;
• clients subscribing to broadly shared destinations;
• RabbitMQ connections and channels being created too frequently. That structure could appear to work because every WebSocket session existed inside the same application process. Even broad delivery eventually reached the process that owned every connected client. The weakness became visible when the service had to support multiple tenants and several WebSocket servers. Broadcasting every event to every client would create unnecessary browser traffic, while a single application server would remain a scaling and failure boundary. I first addressed the most obvious problem: client-level targeting.

First Improvement: Route by User ID

The initial subscription model allowed clients to receive data they did not need. I changed the WebSocket destinations so that a user subscribed to a destination containing its own identifier. A simplified destination looked like this:

/topic/events/{userId}
/topic/updates/{userId}

The publisher also attached the target user ID as the routing key. This meant the application no longer treated every connected browser as a recipient. Conceptually, the flow changed from:

event -> shared destination -> every client

to:

event -> user routing key -> matching user destination

I also changed consumer initialization so it happened when the server started instead of repeatedly registering consumers for client requests, and reused RabbitMQ resources instead of creating a new connection and channel for every send. With one WebSocket server, the improved flow behaved correctly. If user u-104 was connected and subscribed, the server forwarded the event. If no client subscribed to that destination, no browser traffic was generated. The design still contained a hidden assumption: the consumer that received the RabbitMQ message also had the target WebSocket session.

The Failure Appeared with Multiple WebSocket Servers

Consider two WebSocket servers behind a load balancer:

User u-104 -> WebSocket Server A
User u-205 -> WebSocket Server B

Both servers consume from the same RabbitMQ queue:

                       +-> Consumer on Server A
Publisher -> Queue ----|
                       +-> Consumer on Server B

RabbitMQ documents that active consumers on the same queue normally receive deliveries in round-robin fashion. That behavior is appropriate for distributing independent jobs across workers. It does not mean that every consumer receives a copy. See the official RabbitMQ consumer documentation. This created the failing sequence:

1. an event targeting user u-104 was published;
2. the shared queue accepted the event;
3. RabbitMQ delivered it to the consumer on Server B;
4. Server B acknowledged and processed the event;
5. Server B searched its local WebSocket sessions;
6. user u-104 was connected to Server A, so Server B had no matching session;
7. the browser received nothing. From RabbitMQ's perspective, the delivery succeeded. From the user's perspective, the message disappeared. Retries would not correct the design. The event was not failing broker processing; it had reached a valid consumer that lacked the final in-memory destination.

Root Cause: Two Different Routing Questions Were Mixed Together

The architecture had two independent routing questions:

1. Which user should receive this event?
2. Which WebSocket server currently owns that user's connection? Routing by user ID answered the first question. A shared queue with competing server consumers ignored the second. This is a common boundary mistake in real-time systems. RabbitMQ knows exchanges, queues, bindings, consumers, and routing keys. It does not automatically know that a browser session is stored inside one specific Spring Boot process. WebSocket session ownership is typically local, ephemeral state:

server process -> session registry -> connected browser

When several server processes exist, a broker topology must either route to the owning process or send a copy to every process that may own the session.

The Revised Topology

I changed the topology so each WebSocket server had its own queue and consumer. The queues were connected through an exchange scoped to the tenant, while the user ID remained the routing key used by the application. A simplified version looks like this:

Publisher -> Tenant Exchange -+-> Queue A -> WebSocket Server A
                             +-> Queue B -> WebSocket Server B

Routing key: user.{userId}

The important change is not merely adding more queues. It is changing the delivery semantics:

• one queue with multiple consumers: one message is distributed to one competing consumer;
• multiple bound queues: each matching queue receives its own routed copy. RabbitMQ exchanges route messages to queues through bindings. A direct exchange uses exact routing-key equality, while a topic exchange supports segmented patterns. The precise exchange type should follow the application's routing requirements; the official behavior is described in the RabbitMQ exchange documentation. In this system, tenant scoping and user targeting served different purposes:
• the tenant identifier separated independent deployments and routing domains;
• the server-specific queue ensured every WebSocket server received a copy it could evaluate;
• the user ID identified the final application recipient;
• only the server that held the matching session forwarded the event to the browser. The other servers received the routed message but found no matching local subscription and stopped there. That creates more broker-to-server deliveries than direct owner routing, but it restores correctness without requiring a distributed WebSocket session directory.

Why Not Create One Queue per User?

One alternative was to create a RabbitMQ queue for every connected user. That can map the broker topology more directly to recipients, but it also ties queue lifecycle to short-lived client sessions. Before choosing it, I would need clear answers for:

• how queues are declared and deleted during reconnects;
• how stale queues are detected;
• whether a user can connect from multiple browser tabs or devices;
• whether messages should survive a disconnected session;
• how many simultaneous users the broker must support;
• how queue metrics and permissions remain manageable. For this system, server-level queues plus user-level application routing were a better operational boundary. The number of application servers was far smaller and more stable than the number of client connections. A more advanced design could maintain a distributed registry such as:

userId -> websocketServerId

and route directly to the owning server. That reduces fan-out but adds registry consistency, expiration, reconnect races, and failure recovery. It was unnecessary for the first reliable multi-server design.

Testing the Topology, Not Just the Handler

The first improvement passed because all tests ran against one server. The failure required at least two active consumers and a target session attached to only one of them. The minimum useful test matrix was:

1. the application accepted the input event;
2. RabbitMQ routed a copy to the expected queues;
3. only the server with the matching WebSocket subscription sent a frame to the browser. That distinction prevented a broker acknowledgment from being mistaken for end-to-end delivery.

Operational Checks That Matter

The revised topology needs monitoring at each boundary.

RabbitMQ

• queue count per tenant;
• active consumer count per server queue;
• ready and unacknowledged messages;
• unexpected queue churn;
• bindings and routing keys;
• unroutable publishes.

WebSocket servers

• active sessions by server;
• subscriptions by destination;
• events received from RabbitMQ;
• events forwarded to WebSocket clients;
• events skipped because no local session exists;
• reconnect and duplicate-subscription counts.

End-to-end correlation

A message identifier should be logged at publication, queue consumption, and WebSocket forwarding. Without a shared identifier, “RabbitMQ received it” and “the browser received it” are easy to confuse.

What I Would Keep from This Design

The main lesson was not “use more exchanges” or “use more queues.” It was to make delivery semantics explicit. A queue shared by several consumers is a work-distribution mechanism. A WebSocket notification may require fan-out because only one process owns the target connection. Those are different communication models. Before scaling a broker-backed WebSocket service, I now ask:

• Is this message a job that exactly one worker should process?
• Is it an event that several application instances must observe?
• Where does the target WebSocket session live?
• Can the broker route directly to that owner?
• If not, is controlled fan-out acceptable?
• What evidence proves delivery beyond the broker? The original single-server system hid these questions. Adding a second server exposed them immediately—and that failure produced a topology that could grow to multiple tenants without depending on one WebSocket process.

References

• RabbitMQ Consumers
• RabbitMQ Exchanges
• RabbitMQ Queues