OLTP-to-OLAP replication pipeline

DesignHard45 minSnowflake and Databricks style

Your task

Design an OLTP-to-OLAP replication pipeline so analysts can run heavy aggregate queries (revenue by region by day, cohort retention) over a 10 TB transactional Postgres database doing 60k writes/sec, without those analytical queries degrading production OLTP latency. Freshness target is under 1 minute. How do you move data, what target store do you pick, and how do you reconcile updates and deletes?

It covers

  • Htap
  • Etl vs cdc
  • Columnar warehouse