Inventory oversells after region partition

On-call incidentHard45 minShopify and Amazon style

Your task

Your inventory service runs active-active across two regions with async bidirectional replication. After a 4-minute cross-region link partition at 03:00, you oversell: an item with 1 unit in stock got sold to two different customers. Dashboards: during the partition, region-A and region-B each independently decremented the same item's stock to 0; after the link healed, replication 'merged' and the count went negative; both regions accepted a checkout for the last unit. No deploy. How do you triage and mitigate?

It covers

  • Dual write
  • Active active
  • Conflict