No prioritization under overload
Your task
Your API gateway fronts a fleet of services during an unexpected 3x traffic surge. Everything degrades uniformly: health checks, login, paid-customer API calls, and a cheap free-tier polling endpoint all slow to multi-second responses and timeouts, so even health checks fail and instances get killed and restarted, worsening the surge. There is no per-tier throttling, every request competes equally. CPU is pinned fleet-wide. You can't add capacity fast enough. How do you triage and stabilize service for what matters most?
It covers
- Load shedding
- Prioritization
- Overload