Code Room
On-callHard
Question
A worker fleet's CPU jumps to 95% across all instances at 18:00 and stays there even after the original problem clears. Useful work output (jobs completed/sec) collapses while CPU stays pinned. A profile shows most CPU in a tight retry loop: serialize request, call a dependency, catch the error, immediately retry with no backoff. The trigger was a brief 90-second blip in that dependency at 17:58; the dependency is healthy again, but CPU never recovered. Job arrival rate is flat. Triage and mitigate.
What a strong answer looks like
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.
Learn the concepts
Loading whiteboard…
Run or narrate your approach, then ask the coach.