Question
Your distributed cache runs primary-replica with an automatic failover daemon (sentinel-style) that promotes a replica if it can't reach the primary. At 03:40 a brief partition isolates the failover daemon (and some clients) from the primary, but the primary itself is healthy and still serving the clients on its side. The daemon promotes the replica. Now you have TWO primaries accepting writes for ~3 minutes; clients on each side see different data, and after the partition heals you find conflicting/overwritten keys. How do you triage and mitigate this SPLIT-BRAIN and prevent recurrence?
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.