Code Room
On-callMedium
Question
Four hours after a routine model promotion, the growth team escalates: checkout conversion is down 6% and add-to-cart from the 'related products' carousel is down 18%, but there are no errors, latency is normal, and the model returns 200s. Dashboards: the new model version v37 went to 100% at 10:00; offline eval looked slightly better than v36. Click-through on recommendations dropped at 10:05 and stayed down; the candidate-recall stage looks unchanged but the final ranking scores are more uniform (lower variance) than v36's. How do you triage and respond?
What a strong answer looks like
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.
Learn the concepts
Loading whiteboard…
Run or narrate your approach, then ask the coach.