Thumbnail request coalescing

DesignMedium35 min

Your task

Design request coalescing for an internal image-thumbnail service. On a cache miss, generating a thumbnail takes ~800ms and is CPU-heavy. During traffic spikes, hundreds of concurrent requests for the same uncached image all trigger generation simultaneously, melting the workers. Design a coalescing layer so only one generation runs per (image, size) while the rest wait and share the result, across a fleet of N stateless app servers.

It covers

  • Single flight
  • Dogpile
  • Fan in