Question
Design a SIEM / security-log anomaly pipeline that ingests 4M security events/s from endpoints, network sensors, cloud audit logs, and applications, correlates them to detect intrusions (lateral movement, privilege escalation, data exfil), and alerts within a target of under 60s for high-severity patterns. It must retain raw logs for 1 year for forensics and compliance, support analyst ad-hoc hunting queries over that history, and resist an attacker who tries to disable or tamper with logging to cover their tracks. Cover ingestion/normalization, the streaming detection layer, storage tiers, and tamper-evidence.
Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts — data model, bottlenecks, consistency, failure modes — and name the trade-offs you are making.