Code Room
On-callHard
Question
Your JVM search service degrades under a gradual traffic increase: p99 climbs and a few instances become briefly unresponsive, then recover, in a rolling pattern. CPU shows frequent spikes to 100% but the request rate that triggers it is only ~20% above last month. GC logs show full GCs every few seconds with multi-hundred-millisecond pauses, heap usage sawtoothing near the max, and old-gen filling faster than it's collected. A recent change added a large per-request in-memory result cache. How do you triage and mitigate?
What a strong answer looks like
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.
Learn the concepts
Loading whiteboard…
Run or narrate your approach, then ask the coach.