Train-serve skew in feature computation
Your task
After a new pricing-model release, the model's online performance is much worse than its glowing offline eval: predicted prices are systematically biased low and the business sees margin erosion, though there are no errors. Investigating, you find the offline training pipeline computes 'normalized_demand' with a 7-day rolling window in a batch Spark job, while the online serving path computes the same feature in a separate Java service, and the two implementations disagree (the online one uses a different window boundary and unit). Walk through how you confirm the issue, mitigate, and prevent it.
It covers
- Train serve skew
- Feature parity
- Preprocessing
- Rollback