Network flow telemetry collector

DesignHard45 minCisco style

Your task

Design a network-telemetry / flow-record collector that ingests NetFlow/IPFIX/sFlow from 50,000 routers and switches, peaking at 5M flow records/sec over UDP. It must handle lossy UDP ingest, template-based decoding, high-cardinality aggregation (top talkers, per-prefix bytes), and serve both real-time anomaly detection and historical queries. How do you ingest, decode, aggregate, and store at this rate?

It covers

  • Netflow
  • Ipfix
  • Udp ingest
  • Aggregation