Code Room
On-callHard
Question
You run a 3-node self-managed PostgreSQL cluster with a custom Patroni-style failover. At 02:14 customers report some writes 'disappearing' on refresh. Dashboards: two nodes (A and B) both report themselves as primary; A is in us-east-1a, B in us-east-1c; the etcd DCS shows intermittent connectivity between AZs starting 02:09; both A and B are accepting writes and each has a different, diverging WAL position. A network team is mid-incident on an inter-AZ link. How do you triage and mitigate?
What a strong answer looks like
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.
Learn the concepts
Loading whiteboard…
Run or narrate your approach, then ask the coach.