WebSocket Architecture With Node.js, Redis, and Queues

The standard tutorial for Node.js WebSockets involves installing socket.io, writing twenty lines of code in a single server.js file, and watching a chat message instantly appear in two browser windows. It feels like magic. However, when you deploy this single-server code to a production environment that requires horizontal scaling (running multiple servers behind a load balancer), the magic breaks immediately. Users stop receiving messages.

To build a production-grade real-time application, you must decouple the WebSocket connection layer from the message distribution layer and the business logic layer. This article details the standard architecture for scaling Node.js WebSockets.

The Problem: The Stateful Server

HTTP is stateless. If you make an API request to Server A, and your next request goes to Server B, it doesn't matter; both servers can query the central database and return the correct data. WebSockets are stateful. The TCP connection is physically tied to a specific server memory space.

If User A connects to Node Server 1, and User B connects to Node Server 2, they cannot communicate natively. When User A sends a message intended for User B, Server 1 receives it, checks its local memory for User B's connection, finds nothing, and the message dies.

The Solution: Redis Pub/Sub

To bridge the gap between servers, we introduce a central nervous system: a Redis database using the Publish/Subscribe (Pub/Sub) pattern. Redis is an in-memory datastore capable of handling millions of messages per second with microsecond latency.

The Architecture Flow:

1. Subscription: When Node Server 1 and Node Server 2 boot up, they immediately connect to Redis and subscribe to a specific channel (e.g., channel:chat_messages).
2. Publishing: User A (on Server 1) sends a message: "Hello User B".
3. Server 1 receives the WebSocket frame. Instead of trying to find User B locally, Server 1 publishes the message payload to Redis on channel:chat_messages.
4. Distribution: Redis instantly pushes this payload to all subscribed servers (Server 1 and Server 2).
5. Emission: Server 2 receives the payload from Redis, checks its local memory, finds User B's active WebSocket connection, and emits the message to them.

If you are using Socket.io, you do not need to write this Pub/Sub logic manually. You install the @socket.io/redis-adapter, provide it a Redis connection, and Socket.io handles the cross-server broadcasting entirely under the hood. You simply call io.to('room_name').emit(), and the adapter ensures it reaches the right users, regardless of which server they are connected to.

Handling Heavy Workloads: The Role of Queues

A common architectural failure is putting heavy database writes or third-party API calls directly inside the WebSocket event listener.

// BAD ARCHITECTURE
socket.on('document_update', async (data) => {
  // Blocking the event loop for other concurrent users
  await heavyDatabaseUpdate(data);
  await generatePDF(data); 
  io.emit('document_saved');
});

Node.js is single-threaded. If the generatePDF function takes 2 seconds of CPU time, no other WebSocket messages can be processed during those 2 seconds. The entire real-time system freezes for all users on that server.

The Queue Architecture Flow:

1. User A sends a document_update WebSocket event.
2. The Node.js server receives it and immediately adds a job payload to a Redis-backed queue (like BullMQ).
3. The Node.js server immediately responds to User A: socket.emit('update_received_processing'). The event loop is free in less than 5 milliseconds.
4. A completely separate Node.js process (a Worker server) picks up the job from BullMQ, connects to the database, and generates the PDF.
5. When the Worker finishes, it publishes a message to Redis Pub/Sub: { action: 'pdf_ready', userId: 123 }.
6. The WebSocket server receives the Pub/Sub message from Redis and emits the final pdf_ready event to the specific user.

Ensuring Delivery and Ordering

WebSockets guarantee the order of messages over the network (because they use TCP), but they do not guarantee order through your asynchronous Node.js architecture. If User A types "Hello" and then "World" rapidly, and Server 1 publishes both to Redis, network jitter could cause the database worker to process "World" before "Hello".

To solve this, the client must generate a UUID for every message it sends and attach a timestamp. The database must use UPSERT operations or sequence numbers to reject older messages if they arrive out of order, ensuring the final database state is consistent regardless of processing speed.

Architecting a real-time system is rarely about the WebSocket connection itself; it is about managing the distributed state across multiple servers and workers. If you need a scalable architecture for your Node.js application, visit my Node.js development services page.