Rate limiter with burst smoothing
Your task
Design an API rate-limiter service fronting a public API gateway handling ~80K req/sec across a fleet of ~40 gateway instances. Limits are per-API-key and tiered (e.g. 100 req/min free, 10K req/min enterprise) plus a global per-key burst limit. Requirements: enforce limits accurately across all gateway instances (not per-instance), add <2ms to each request, fail-open if the limiter backend is down (don't take the API offline), and return correct rate-limit headers (remaining, reset). Smooth bursting is preferred over hard cliffs.
It covers
- Rate limiting
- Distributed counters
- Algorithms
- Low latency