On-callMediumoc-g044

Subject Tail latencyLevel Mid–Senior~30 minCommon in Networking & APIs interviewsIndustries Technology, Software development

Question

A Java/JVM service runs behind a round-robin load balancer with rolling deploys, 40 instances. During and just after each rolling deploy, overall p99 latency spikes to 5-10x normal for a few minutes per instance, then settles — p50 is barely affected. Traces show the slow requests are landing on instances that were started seconds ago; the same endpoint on a long-running instance is fast. CPU on the fresh instances is briefly high. There's no error spike. How do you triage, and how do you stop fresh instances from poisoning the tail?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Learn the concepts

Diagram & narrate the incident

Loading whiteboard…

Run or narrate your approach, then ask the coach.