Shared dependency saturates on surge

On-call incidentHard30 minNetflix style

Your task

A big live event drives a 6x surge to your video-start API. That service autoscales fine and stays healthy, but starts failing because a shared entitlement service it calls (also used by 9 other teams) saturates: the entitlement service's DB connection pool is exhausted and it returns 503s, which cascade into your video-starts AND degrade the 9 unrelated teams. Your service is already correctly caching entitlements for 60s. How do you triage and mitigate, balancing your incident against the shared-dependency blast radius?

It covers

  • Dependency saturation
  • Shared downstream
  • Fan out