Code RoomBackdoor under secure aggregation
FrontierPrep Room Coding #4979

Backdoor under secure aggregation

System designDistributed systemsSenior–Staff~75 min

You run training across a large device fleet. Secure aggregation is deployed and it works: the server sees the sum of a round's updates and never an individual one, and each client proves in zero knowledge that its update respects the clipping norm. That property is why the deployment was approved, so it is not available to trade away.

Some fraction of the fleet answers to one party. Its devices submit well formed updates, inside the norm bound, chosen so the global model acquires a behavior nobody asked for on a narrow slice of inputs that party selects.

Clipping already gives you a displacement bound: a coalition holding a given share of a round can move the aggregate by at most that share of the clip norm. The bound is real and it is not the bound you need. A small displacement in a well chosen direction is enough to install a narrow behavior, and nothing connects the norm you bounded to the behavior you care about. That gap is the whole problem, and your answer has to work inside it.

Three exits are closed. You may not open individual updates, because that is the property you shipped. You may not assume a server side validation set representative of the population, because if you had one you would not be training this way. You may not assume device attestation removes the coalition, because the devices can be honest hardware whose local data was chosen.

Hand in the round protocol and its admission rules, a bound on behavioral change as a function of coalition share and clip norm, or the argument that no such bound is available at this granularity together with what you substitute for it, the aggregate level signals you watch across rounds and what each one would miss, the utility bill paid by honest clients with the slowest and most unusual data, and the experiment that would show your defense responds to a coalition rather than to ordinary heterogeneity. Ties break toward the design reporting a wide honest bound over one that assumes the coalition is loud.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.

Clarify8:30 left
Estimate8:30 planned
Design25:00 planned
Deep dive20:00 planned
Failure13:00 planned
0:00
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.