Real-time feature aggregation

DesignHard45 minUber and DoorDash style

Your task

Design the streaming feature pipeline that computes real-time aggregate features for a ride-hailing surge/fraud model: features like 'ride requests in this geohash in the last 5 minutes', 'driver cancellations in the last hour', updated continuously and read at inference with single-digit-ms latency. Events arrive out of order and occasionally late by minutes. Design the streaming aggregation + serving so the online feature values are correct under out-of-order/late events and don't drift from the offline-recomputed truth.

It covers

  • Streaming features
  • Real time aggregation
  • Windowing