Fact table backfill without downtime

DesignHard50 minNetflix style

Your task

A bug in a transformation under-counted user watch-minutes for the last 90 days across a partitioned fact table (~3 TB/partition-day, 90 days). You must backfill the corrected values without taking the table offline for live dashboards, without double-counting, and without saturating the shared cluster that also runs daily pipelines and ad-hoc analyst queries. Design the backfill: how you reprocess 90 days correctly, how you swap in corrected data safely, and how you avoid starving production workloads.

It covers

  • Reprocessing
  • Idempotency
  • Throttling