Code Room
On-callHard
Question
A DynamoDB table stores user documents with a TTL attribute `expires_at` so abandoned drafts auto-delete after 90 days. A backfill ran yesterday to add `expires_at` to ~12M legacy rows that lacked it. Overnight, support reports active users' live documents *vanishing* — not drafts, real saved work — and the count keeps growing hour over hour. Dashboards: no error spike, no manual deletes in CloudTrail, but the table's item count is dropping and the TTL-deletes metric is elevated. How do you triage, stop the loss immediately, and recover the deleted items?
What a strong answer looks like
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.
Learn the concepts
Loading whiteboard…
Run or narrate your approach, then ask the coach.