Scan-resistant cache eviction

DesignMedium35 min

Your task

Your in-memory cache (fixed 64GB) is showing a low hit rate even though the popular working set easily fits in memory. Investigation shows occasional large scans (a batch job, a crawler) pull in thousands of one-hit-wonder keys that evict the genuinely hot keys under plain LRU, cache pollution. Design an eviction/admission policy that resists scan pollution, keeps the genuinely hot working set resident, and adapts as the popularity distribution shifts over time. Discuss how you'd measure that it's working.

It covers

  • Eviction
  • Admission
  • Working set