WebSocket gateway fleet
Your task
Design the WebSocket gateway tier that holds 10 million concurrent persistent connections for a real-time app, where any backend service must be able to push a message to a specific user (who may be connected to any of hundreds of gateway nodes, or not connected at all). Connections churn heavily (mobile users dropping in and out), and a deploy of the gateway fleet must not drop everyone at once. How do you route a backend push to the right connection across the fleet, and how do you handle churn and deploys?
It covers
- Connection management
- Scale out
- Reconnection
- Load balancing