WebSocket load balancing

DesignHard45 min

Your task

Design load balancing for a real-time presence/WebSocket service where each user holds a long-lived connection and is pinned to one server that owns their session state. You run thousands of servers, need even connection distribution, and must minimize how many users get disconnected when you add/remove servers during a deploy or autoscale event. Explain the routing scheme and how you handle a server dying mid-session.

It covers

  • Consistent hashing
  • Stateful routing
  • Rebalance