Code RoomEtcd cluster writes failing
HardPrep Room Coding #2587

Etcd cluster writes failing

On-callDistributed systemsSenior–Staff~45 min

Your config service is backed by a 5-node etcd cluster across 3 AZs (2/2/1). At 04:00 all writes start failing with 'no leader' / context-deadline-exceeded, though reads still work. Dashboards: an AZ hosting 2 etcd nodes lost network at 03:58; the remaining cluster has 3 of 5 nodes; leader elections are flapping; raft proposals time out; clients that need to WRITE config are blocked. The 2-node AZ isn't coming back soon. How do you triage and mitigate?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

0:00 of about 45 min
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.