Lead calibration drifts over-confident
Your lead-scoring model outputs a calibrated probability that sales uses with a fixed 0.7 threshold to decide which leads to call. A 'calibration drift' alarm fires: a reliability-diagram check shows predicted probabilities are now systematically over-confident — leads the model says are 0.8 convert at only ~0.55 in reality — and this has worsened over the past three weeks. AUC/ranking quality, however, is essentially unchanged (the model still orders leads well). No model deploy in 50 days. Dashboards: the overall positive base rate has slowly fallen (market softened) while feature distributions are roughly stable. How do you triage and respond?
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.