Code RoomFeature and prediction drift detection
HardPrep Room Coding #3836

Feature and prediction drift detection

System designML systemsReliability & on-callSenior–Staff~40 min

Design an ML feature-monitoring and observability system that watches features and predictions across ~200 production models. For each model you must detect, within minutes-to-hours, when an input feature's distribution shifts, when a feature pipeline silently starts emitting nulls or stale values, and when prediction distributions drift relative to training. The system ingests roughly 500k predictions/sec in aggregate, must support both real-time alerting and historical investigation, and most models have delayed or no ground-truth labels, so you can't rely on accuracy as the primary signal.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.

Clarify4:30 left
Estimate4:30 planned
Design13:30 planned
Deep dive10:30 planned
Failure7:00 planned
0:00
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.