Request coalescing across CDN hierarchy

DesignHard40 minNetflix style

Your task

A CDN with 200 PoPs serves a popular live-event video manifest that's regenerated every 2 seconds. On each refresh, every PoP misses and hits the origin; with 200 PoPs each running many cache nodes, the origin sees a request storm every 2 seconds. Design request coalescing across the whole hierarchy so the origin sees roughly one request per refresh interval, not thousands. Cover within-PoP coalescing, cross-PoP coalescing (origin shield), and what happens on the failure/timeout path so a slow origin response doesn't pile up coalesced waiters.

It covers

  • Single flight
  • Origin shield
  • Tiered