Code RoomModel monitoring without ground truth
HardPrep Room Coding #3218

Model monitoring without ground truth

System designReliability & on-callSenior–Staff~40 min

Design a production monitoring system for a credit-risk model that decides loan approvals. The catch: the true label (did the borrower default?) only arrives 6-18 months after the prediction, so you cannot measure accuracy in real time. The business needs early warning if the model starts degrading or behaving unfairly. Design what you monitor in the absence of ground truth, how you detect drift, and how you'd catch a silent failure (e.g. an upstream feature pipeline breaks) within minutes, not months.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.

Clarify4:30 left
Estimate4:30 planned
Design13:30 planned
Deep dive10:30 planned
Failure7:00 planned
0:00
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.