Streaming windowed aggregations

DesignHard50 minPayPal style

Your task

Design the streaming-aggregation engine of a feature store that computes real-time windowed features ('count of logins from this account in the last 5 minutes', 'sum of transfer amounts in the trailing hour') for an account-takeover model, where these features must be correct under out-of-order and duplicate events and must read at <5ms online. Walk through the windowing and aggregation model, how you guarantee a duplicate event isn't double-counted, and how the same aggregation stays consistent between the streaming online path and the batch path used to generate training data.

It covers

  • Streaming aggregation
  • Exactly once
  • Windowing
  • Online offline parity