Wallet balance corruption under concurrency
Your task
An accounting reconciliation flags that aggregate 'credits remaining' across wallets is drifting LOWER than the sum of individual debits/credits should allow, money is quietly disappearing from balances under high concurrency. Dashboards: no errors; the wallet service reads a balance, computes the new value in app code, and writes it back (UPDATE wallets SET balance = ? WHERE id = ?); during traffic spikes, concurrent debits on the same hot wallet (shared team accounts) interleave; the DB is Postgres at READ COMMITTED. There's no row lock or atomic decrement. How do you triage this corruption, stop the silent balance loss, and recover the lost amounts?
It covers
- Concurrent update
- Lost update
- Read modify write
- Counters