WebSocket gateway fleet

DesignHard50 minMeta and Salesforce style

Your task

Design the WebSocket gateway tier that holds 10 million concurrent persistent connections for a real-time app, where any backend service must be able to push a message to a specific user (who may be connected to any of hundreds of gateway nodes, or not connected at all). Connections churn heavily (mobile users dropping in and out), and a deploy of the gateway fleet must not drop everyone at once. How do you route a backend push to the right connection across the fleet, and how do you handle churn and deploys?

It covers

  • Connection management
  • Scale out
  • Reconnection
  • Load balancing