Code Room
On-callHardoc-g052
Subject RollbackLevel Senior–Staff~45 minCommon in Reliability & on-call interviewsIndustries Technology, Software development

Question

v300 of the user service deployed at 11:00 with a migration that renamed the column 'phone' to 'phone_number' and the app now writes to 'phone_number'. At 11:20 you discover v300 has a separate bug corrupting notification preferences and you decide to roll back to v299. But v299's code reads/writes 'phone' which no longer exists, so a naive rollback will 500 on every request. Meanwhile 20 minutes of writes have only landed in 'phone_number'. Walk me through how you safely get back to a healthy state.

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Diagram & narrate the incident
Loading whiteboard…
Run or narrate your approach, then ask the coach.