Question

Design a concurrent cache with single-flight (dogpile / thundering-herd prevention) in front of an expensive backend (e.g. a 300ms recommendation computation) serving 100k req/sec across many app servers. Constraints: when a hot key expires, the backend must be called once, not by every concurrent miss, stale-but-fast responses are acceptable briefly, and a slow recompute must not cascade into backend overload. Describe the cache layout, how concurrent misses for the same key are coordinated, and the recompute path.