Dynamic fair-share rate limiter

DesignHard45 min

Your task

Design a fair-share rate limiter for a shared backend resource (a database connection pool of 500 connections) used by 2000 tenants. Each tenant has no fixed quota; instead, when the resource is uncontended a tenant may use a lot, but when many tenants are active, capacity must be shared fairly so one heavy tenant can't starve others (the noisy-neighbor problem). Static per-tenant limits waste capacity off-peak and don't adapt. Design the dynamic fair-sharing mechanism, how you detect and react to contention in real time, and how you bound the worst-case latency a starved tenant sees.

It covers

  • Distributed
  • Fairness
  • Tenant isolation