Code Room
On-callHard
Question
A primary Postgres instance crashed (host failure) and your HA controller auto-promoted an async streaming standby. Service recovered in ~40s. But now support is getting reports that a handful of orders placed in the last minute before the crash 'disappeared' — they were charged but the order doesn't exist in the DB. Reconciliation against the payment processor confirms ~30 orders are missing from the new primary. The promoted standby was healthy and ~2s behind at promotion time. Triage, mitigate the user impact, and prevent recurrence.
What a strong answer looks like
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.
Learn the concepts
Loading whiteboard…
Run or narrate your approach, then ask the coach.