Code Room
On-callHardoc-g057
Subject Canary failureLevel Senior–Staff~40 minCommon in Reliability & on-call interviewsIndustries Technology, Software development

Question

v140 of the cart service auto-promotes from canary to 100% at 12:00 after a clean 30-minute canary. By 12:25 support reports a steady trickle of users whose carts 'reset to empty' after adding items. Overall add-to-cart success is 99.95%; only ~0.3% of sessions are affected. The canary used per-request random routing (a request could hit canary or baseline independently). v140 changed the cart session-token format. Triage and explain why the canary passed, then mitigate.

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Diagram & narrate the incident
Loading whiteboard…
Run or narrate your approach, then ask the coach.