Real-time engagement features

DesignMedium40 minAmazon style

Your task

Design the real-time engagement-feature pipeline that feeds a search ranker: the ranker uses features like 'this item's click-through rate over the last 1 hour / 24 hours / 7 days' and 'this item's add-to-cart rate today', and these counters must update within seconds of user events so a suddenly-popular item ranks up quickly. You have millions of items and 200k engagement events/sec. How do you compute and serve these time-windowed counters at low latency, and how do you keep them from being gamed or skewed by a single burst?

It covers

  • Feature logging
  • Real time counters
  • Freshness