Fact table backfill without downtime
Your task
A bug in a transformation under-counted user watch-minutes for the last 90 days across a partitioned fact table (~3 TB/partition-day, 90 days). You must backfill the corrected values without taking the table offline for live dashboards, without double-counting, and without saturating the shared cluster that also runs daily pipelines and ad-hoc analyst queries. Design the backfill: how you reprocess 90 days correctly, how you swap in corrected data safely, and how you avoid starving production workloads.
It covers
- Reprocessing
- Idempotency
- Throttling