On-callHardoc-g259

Subject Latency spikesLevel Senior–Staff~40 minCommon in Reliability & on-call interviewsIndustries Technology, Software development

Question

A memory-bandwidth-heavy in-memory analytics service was migrated to a new dual-socket bare-metal host with more total cores and RAM, but per-query latency got ~25% worse and is now erratic, with p99 swinging run-to-run. CPU is not saturated. perf shows high memory-access latency and a large fraction of remote-socket memory accesses; the old single-socket box had none. The process is not pinned and the OS freely migrates threads across both sockets. How do you triage and mitigate?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Learn the concepts

Diagram & narrate the incident

Loading whiteboard…

Run or narrate your approach, then ask the coach.