Global distributed rate limiter

DesignHard45 minStripe style

Your task

Design a distributed rate limiter for a public API gateway running on 200 edge nodes that must enforce per-API-key limits (e.g. 1000 req/min) globally, not per-node. Constraints: 500k req/sec aggregate, the limit decision must add under 1ms to the request path, and overshoot of the global limit should be small and bounded. Explain where counters live, how concurrent nodes coordinate a shared budget, and what happens when the coordination store is briefly unreachable.

It covers

  • Token bucket
  • Sharding
  • Consistency