Postgres CDC to warehouse
Your task
Design change-data-capture replicating a 4 TB Postgres OLTP database (3k writes/sec, frequent schema changes from a fast-moving product team) into a warehouse and a search index, with end-to-end lag under ~30s. Constraints: the search index must reflect deletes promptly, the warehouse needs full history (every version of a row), and you cannot put meaningful extra load on the OLTP primary. Design the capture mechanism, how you do the initial load without blocking, and how you keep it correct across schema changes.
It covers
- Log based cdc
- Ordering
- Initial snapshot