ML experimentation platform
Design an experimentation platform for evaluating ML model changes (ranking, recommendations, pricing) across a product with 100M users and dozens of teams running concurrent experiments. It must assign users to variants consistently, prevent overlapping experiments from confounding each other, compute statistically valid results with guardrail metrics, and let a team safely ramp or roll back a model. Cover assignment, the metric pipeline, and how you handle interaction between experiments.
What a strong answer looks like
Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.
Clarify5:30 left
Estimate5:30 planned
Design16:30 planned
Deep dive13:30 planned
Failure9:00 planned
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.
Run or narrate your approach, then ask the coach.