Question

Design the load-balancing layer for a distributed cache fleet (say 500 cache servers) where you want request affinity so the same key lands on the same server (to maximize cache hit rate) but you must avoid hot servers when a few keys go viral. Plain consistent hashing pins a viral key to one overloaded node. Design a scheme that keeps affinity for the long tail but spreads load for hot keys, handles servers joining/leaving with minimal key remapping, and explain how you detect and react to a node approaching saturation in real time.