Question
Design an API gateway that fronts ~400 internal microservices for a B2B SaaS platform handling 150k requests/sec at peak with a p99 added-latency budget of 8ms. The gateway must terminate TLS, authenticate JWTs, enforce per-tenant rate limits, route by path/host to the right upstream, and support canary weights and blue/green cutovers without dropping in-flight requests. Route configuration changes (a new service, a weight shift) must propagate to all gateway nodes within a few seconds, and a single misconfigured route must not be able to take down the whole fleet. Walk through the components, the routing-table model, and how you keep config consistent across nodes.
Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts — data model, bottlenecks, consistency, failure modes — and name the trade-offs you are making.