Dual-write flag causes read-write skew
Your task
A flag 'write_to_new_store' has been ramping for the document service: it dual-writes edits to both the old store and a new store. At 5% for a week, fine. At 16:00 it's bumped to 50%. Within the hour, users report documents 'losing recent edits' on reload. The new store's write success is 100% and so is the old store's. But the flag is evaluated per-request, and a given document's writes can land on either store depending on which request evaluated the flag true. Reads always come from the old store. Triage, explain the corruption, then mitigate.
It covers
- Data corruption
- Dual write
- Rollout scope