Stream-batch reconciliation
Your task
A financial reporting warehouse aggregates transactions from a streaming pipeline (real-time) and a separate batch settlement file from the payment processor (authoritative, arrives next day). The two frequently disagree: the streaming total and the settlement total differ by small amounts due to dropped/late events, retries, currency rounding, and timezone cutoffs. Design an automated reconciliation system that detects, classifies, and surfaces these discrepancies so finance can trust the numbers and auditors can see a clean trail.
It covers
- Reconciliation
- Completeness
- Source of truth