Origin servers saturate

On-call incidentMedium35 min

Your task

Your origin servers suddenly see a 5x jump in request rate and CPU saturation, with origin latency climbing and some 5xx errors, even though end-user traffic (requests hitting the CDN) is flat. CDN dashboards show the cache hit ratio dropped from ~96% to ~70% over 30 minutes, and origin-facing requests spiked. A deploy 40 minutes ago changed how the app sets cache headers and added a request-correlation query parameter to asset URLs for tracing. An 'origin shield' tier sits between edge POPs and origin. Triage and fix.

It covers

  • Cdn
  • Origin shield
  • Cache miss