Regional quota exhaustion on surge

On-call incidentHard30 minAmazon style

Your task

You run the same service in three regions behind latency-based DNS. During an outage at a peer CDN, traffic shifts and us-east-1 takes a 3x surge while eu-west-1 and ap-southeast-1 stay flat. us-east-1 fails to scale: new instances launch but a chunk never reach service and the ASG log shows a service-quota error. The other two regions are healthy and well under the same nominal limits. Your global capacity dashboard (which aggregates across regions) shows plenty of headroom. How do you triage and mitigate?

It covers

  • Per account quota
  • Single region
  • Regional skew