Design stateful windowed stream aggregation
Your task
Design a stateful stream-processing pipeline that computes per-key windowed aggregations (e.g. 5-minute rolling revenue per merchant) over an event stream where events arrive out of order and late (mobile clients buffer offline for minutes/hours). 300K events/sec, results must be correct despite late data, and the operator state (per-key windows) can be large and must survive crashes. Emit results with bounded latency.
It covers
- Windowed aggregation
- Watermarks
- Late data