Code Room
On-callHard
Question
Your distributed query engine scales workers horizontally and normally near-linearly. After onboarding a large analytics customer you added 3x more worker nodes to handle their load, but total throughput barely improved and per-query latency got worse. Worker CPU is moderate. Tracing shows every query, on start and on each stage transition, calls a single central coordinator/metadata service to fetch the schema and assign splits — and that coordinator is now at 100% CPU with a growing request queue. How do you triage, and what's the real fix?
What a strong answer looks like
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.
Learn the concepts
Loading whiteboard…
Run or narrate your approach, then ask the coach.