System designMediumsd-g169

Subject Log aggregationLevel Mid–Senior~35 minCommon in Algorithms & data structures interviewsIndustries Technology, Software development

Question

Design the ingest-time processing stage of a log platform for a regulated fintech: 500GB/day of logs from 300 services in inconsistent formats (JSON, logfmt, raw text). Requirements: parse/normalize into a common schema, redact PII (card numbers, SSNs, emails) before it's ever persisted, drop noisy debug lines to control cost, and route security-relevant logs to a separate compliance store with 7-year retention while app logs keep 30 days. Design the processing pipeline and the routing.

What a strong answer looks like

Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts — data model, bottlenecks, consistency, failure modes — and name the trade-offs you are making.

Learn the concepts

Narrate your design

Loading whiteboard…

Run or narrate your approach, then ask the coach.