Ordered CDC replication over WAN

DesignHard50 min

Your task

Design a cross-region replication pipeline that ships an ordered change stream (CDC) from a primary region's transactional database to a secondary region used for read-scaling and DR, over an unreliable WAN that drops and reorders packets and occasionally has multi-minute outages. Requirements: the secondary must apply changes in an order consistent with the primary's commit order (no applying a child row before its parent / no out-of-order updates to the same key), must not lose or duplicate a change even across a connection reset, and replication lag must be observable so DR failover can make an informed RPO decision. Cover the log/offset model, ordering, exactly-once apply, and lag measurement.

It covers

  • Cdc
  • Log shipping
  • Ordering
  • Exactly once