Cache timing side channel
A serving platform carries tens of thousands of small tenants. A shared cache of computed prefixes and retrieval results is the reason the tail of that population is affordable at all: most of those tenants send few requests, and a private partition per tenant is a cache that never warms in its lifetime. Hits are visible from outside. The first-token latency of a hit and a miss differ by more than any jitter you can afford to add, and any tenant can issue requests as fast as its plan allows.
The property at risk is whether a chosen item, a document, a prompt, a prefix, was recently processed for somebody else. Per-tenant partitioning closes that channel, and its memory cost grows with tenant count, which is the economics you were hired to protect. Partitioning is a legitimate part of your design only if you price what it does to the tail and then argue that what stays shared is safe.
Off the table: padding every response to a worst case, since the legitimate spread of first-token latency runs over orders of magnitude and paying the worst case removes the reason the cache exists. Off the table: a rate limit offered as a solution without saying what a patient attacker holding many accounts still learns. Off the table: any leakage claim stated for one observation and never composed over an unbounded adaptive sequence of them.
Hand in the sharing and admission policy: what is shared, with whom, for how long. Hand in the leakage statement, written as an adversary's advantage in deciding whether a target processed a chosen item, and composed over n adaptive probes rather than one. Hand in the hit-rate bill, specifically for the smallest tenants. Name the channel your statement does not cover, from the batch you share, the queue you wait in, the retrieval backend, and your own decode rate. Hand in a harness measuring how many probes an attacker actually needs, and the number at which you would stop sharing.
Ties break toward a statement that survives repetition over one that sounds stronger and holds once.
Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.