Log live tail streaming
Your task
Design the real-time 'live tail' feature of a log platform: an engineer opens a filtered stream (e.g. service=checkout AND level=error) and sees matching log lines appear within ~1s of being emitted, while terabytes/day are still being ingested for storage. During an incident, hundreds of engineers open live tails with different filters simultaneously. Design how live matching works without scanning storage, how you fan results out to many concurrent subscribers, and how you protect ingest.
It covers
- Live tail
- Fan out
- Streaming query