System designMediumsd-g373

Subject Ml systemsLevel Mid–Senior~40 minCommon in ML systems interviewsIndustries Technology, Software development

Question

Design a shadow-deployment and validation harness for promoting a new fraud-scoring model to production, where you can't just A/B it freely because a worse model means real fraud losses or wrongly-blocked legitimate payments — both costly. You need confidence the new model behaves correctly on live traffic before it makes any real decision. Walk through how shadow mode works, what you compare to gain confidence, and how you close the gap between offline-eval results and live behavior.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts — data model, bottlenecks, consistency, failure modes — and name the trade-offs you are making.

Learn the concepts

Narrate your design

Loading whiteboard…

Run or narrate your approach, then ask the coach.