Feature pipeline quality gates
Your task
An ML feature pipeline computes ~600 features daily that feed recommendation and ranking models in production. Twice this year a silent upstream change (a field went 100% null, a currency unit flipped) corrupted features, the model degraded for days, and revenue dropped before anyone noticed because nothing 'errored'. Design a data-quality and observability system that catches these silent corruptions automatically, distinguishes real anomalies from legitimate shifts (e.g. Black Friday traffic), and decides whether to block a bad feature from reaching the model.
It covers
- Anomaly detection
- Circuit breaker
- Observability