TLS-terminating ingress router

DesignHard45 minShopify style

Your task

Design a TLS-terminating ingress that serves 500k custom customer domains (vanity hostnames) on shared infrastructure at 120k requests/sec, routing each connection to the right backend by SNI. It must present the correct per-domain certificate (issued/renewed automatically via ACME), terminate TLS with p99 handshake overhead under 10ms, support both HTTP/1.1 and HTTP/2, and route based on SNI before any application-layer data. A misissued or expired cert for one domain must not affect others. Walk through the ingress architecture, the cert/SNI model, and the central trade-off.

It covers

  • Sni routing
  • Tls termination
  • Cert management