Security event correlation pipeline
Your task
Design a SIEM / security-log anomaly pipeline that ingests 4M security events/s from endpoints, network sensors, cloud audit logs, and applications, correlates them to detect intrusions (lateral movement, privilege escalation, data exfil), and alerts within a target of under 60s for high-severity patterns. It must retain raw logs for 1 year for forensics and compliance, support analyst ad-hoc hunting queries over that history, and resist an attacker who tries to disable or tamper with logging to cover their tracks. Cover ingestion/normalization, the streaming detection layer, storage tiers, and tamper-evidence.
It covers
- Siem
- Anomaly detection
- Log pipeline
- Threat detection