Network flow telemetry collector
Your task
Design a network-telemetry / flow-record collector that ingests NetFlow/IPFIX/sFlow from 50,000 routers and switches, peaking at 5M flow records/sec over UDP. It must handle lossy UDP ingest, template-based decoding, high-cardinality aggregation (top talkers, per-prefix bytes), and serve both real-time anomaly detection and historical queries. How do you ingest, decode, aggregate, and store at this rate?
It covers
- Netflow
- Ipfix
- Udp ingest
- Aggregation