JVM warmup latency on deploy

On-call incidentMedium35 min

Your task

During a rolling deploy of your JVM service, every freshly deployed instance shows a 60–90 second window of high p99 latency and elevated errors right after it starts taking traffic, then settles to normal. The load balancer adds each new instance to rotation as soon as its /health returns 200, which happens within 2 seconds of process start. Heap and GC look fine after warmup; the spike is only at the start of each instance's life. Caches are empty and the JIT hasn't compiled hot paths yet. How do you triage and stop deploys from causing latency spikes?

It covers

  • Jit warmup
  • Deploy
  • Jvm