Code RoomCoordinator bottleneck on scale
HardPrep Room Coding #2849

Coordinator bottleneck on scale

On-callDistributed systemsSenior–Staff~30 min

Your distributed query engine scales workers horizontally and normally near-linearly. After onboarding a large analytics customer you added 3x more worker nodes to handle their load, but total throughput barely improved and per-query latency got worse. Worker CPU is moderate. Tracing shows every query, on start and on each stage transition, calls a single central coordinator/metadata service to fetch the schema and assign splits — and that coordinator is now at 100% CPU with a growing request queue. How do you triage, and what's the real fix?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

0:00 of about 30 min
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.