Skip to content
AI Engineer Guide.

concept · basics · intermediate · concept 5 of 24

WebSockets

0 of 24 complete

Why it matters

A request-response API cannot push tokens as they are generated, so streaming interfaces reach for a socket that stays open. That solves the direction problem and creates three new ones: the connection can die silently and needs detection and resumption, the server must not write faster than the client can read, and a user connected to one instance cannot receive an event published on another. Each is application protocol work that HTTP hid.

Mental model

It is a phone call rather than a letter: both sides can speak at once, and the call can drop without either noticing immediately. The analogy breaks because a dropped call is obvious to a person, while a dead socket can look open until a write fails.

How it works

Upgrade, frames, heartbeat, resumeInteractive

HTTP until the 101; frames after it. Frames are drawn endpoint to endpoint: the proxy forwards them opaquely, and only its handshake and idle timer are shown. The proxy owns the idle timeout and the server owns the sequence window, so both must agree with the client's heartbeat.

Client

browser

Proxy

idle 30 s

Server

seq window

Frames — the application protocol starts here
Step 1/6 · reject upgrade
2 · 401 · no upgradeerror · Proxy → Client
HTTP/1.1 401 Unauthorized
Content-Type: application/problem+json

{ "code": "ticket_replayed",
  "detail": "ticket t_4c02 was already used" }

No 101, no socket, no frames. Authenticating
at the upgrade makes the rejection an HTTP
status instead of a silent close.

Step 1: reject upgrade. Hover, tap or tab to any message to inspect its payload.

Figure. The upgrade is HTTP and everything after it is the application's protocol. The ticket is checked once, the 101 hands the connection over to frames, a heartbeat that outlives the proxy closes the socket, and the reconnect resumes from the last sequence number instead of a new conversation.

What to notice. The failure is a heartbeat that fits the wrong timeout: the client pings every 45 s while the proxy closes at 30 s, so the socket dies with no close frame and the client reports 1006. Backpressure is bounded per connection and a slow consumer is closed rather than buffered; that path is not drawn. Ticket values, sequence numbers and intervals are illustrative.

Text version

Sequence across Client, Proxy and Server. Step 1, reject upgrade: Client to Proxy GET /ws with a stale single-use ticket, Upgrade websocket, Sec-WebSocket-Key, version 13 and subprotocol v1; Proxy to Client 401 with code ticket_replayed, so no socket is opened. Step 2, upgrade: Client to Proxy the same upgrade with a fresh ticket; Proxy to Client 101 Switching Protocols with Sec-WebSocket-Accept and Sec-WebSocket-Protocol v1. A divider marks the start of frames. Step 3, subscribe: Client to Server a subscribe frame for topic tokens:conv_88 with since 412; Server to Client token frames seq 413 to 419. Step 4, heartbeat: Client to Server a ping control frame every 45 s, outside the proxy's 30 s idle limit and inside the server's 60 s limit; Server to Client a pong. Step 5, idle drop: Proxy to Client no close frame, and the client reads CloseEvent code 1006 with wasClean false, because its 45 s ping interval outlived the proxy's 30 s timeout. Step 6, resume: Client to Server a new socket after backoff with a fresh ticket and a subscribe since 419; Server to Client replayed frames 420 to 424 flagged replay true, then live frame 425. Sequence numbers, not the socket, are the resume point.

  1. Authenticate during the upgrade, and carry the identity on the connection rather than trusting a later message.
  2. Define an application message envelope with a type, an id and an error shape, so the protocol is inspectable.
  3. Send heartbeats and enforce an idle timeout on both ends, and configure the proxy to allow the interval you chose.
  4. Bound the outbound queue per connection and close slow consumers rather than buffering without limit.
  5. Reconnect with exponential backoff and jitter, and resume from a sequence number so missed messages are recoverable.
  6. Publish events through a shared bus so any instance can serve any connection, and keep per-connection state minimal.

Trade-offs

ChoiceGood forCostFails when
WebSocketLow-latency bidirectional trafficCustom protocol, reconnect and fan-out workTraffic is one-shot request-response
Server-sent eventsSimple unidirectional streaming over HTTPOne direction and connection limits per hostThe client must send frequent messages
Long pollingWorks through restrictive proxiesHigher latency and overheadUpdates are frequent
Sticky sessionsSimple fan-outUneven load and lost state on reconnectInstances scale frequently
Trade-off mapInteractive

Good for

Low-latency bidirectional traffic

Cost

Custom protocol, reconnect and fan-out work

Fails when

Traffic is one-shot request-response

  1. Traffic is one-shot request-response
  2. The client must send frequent messages
  3. Updates are frequent
  4. Instances scale frequently

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: WebSocket is good for Low-latency bidirectional traffic, costs Custom protocol, reconnect and fan-out work, and fails when Traffic is one-shot request-response. Server-sent events is good for Simple unidirectional streaming over HTTP, costs One direction and connection limits per host, and fails when The client must send frequent messages. Long polling is good for Works through restrictive proxies, costs Higher latency and overhead, and fails when Updates are frequent. Sticky sessions is good for Simple fan-out, costs Uneven load and lost state on reconnect, and fails when Instances scale frequently

Failure modes

  • Idle disconnect. A proxy closes the connection after a quiet period and the client shows a stalled stream. Debug: compare the proxy idle timeout with the heartbeat interval and check close codes in the client log.
  • Unbounded buffer. A slow client causes the server to buffer messages until memory grows without limit. Debug: chart queued bytes per connection and alert when it exceeds a bound.
  • Lost messages on reconnect. The client reconnects and misses events published during the gap. Debug: compare the last sequence number received with the server's log for that connection.

In production

Monitor open connections, reconnect rate, queued bytes and heartbeat failures per instance. Cap connections per instance and per tenant, and test the reconnect path with a proxy that drops idle connections.

Common misconceptions

People think: WebSockets are always better than HTTP streaming for real-time.

Actually: They add protocol and operational work; a one-way token stream is often simpler over server-sent events.

People think: A ping keeps a connection alive everywhere.

Actually: Intermediate proxies have their own idle timeouts, which must be configured to match the heartbeat.

Interview angle

Cover the upgrade, the application protocol, heartbeats, backpressure and fan-out. Know when a simpler streaming transport is enough. Opening a socket and assuming delivery is reliable is the gap.

Sources