On-callMediumoc-g254

Subject Error rate spikesLevel Mid–Senior~30 minCommon in Reliability & on-call interviewsIndustries Technology, Software development

Question

Your auth-gated API suddenly returns 401s for ~8% of requests, but only from one of three Kubernetes node pools, and only in bursts. The 401s come with 'token not yet valid' (nbf) and occasionally 'token expired' errors validating short-lived (60s) JWTs. The affected pods were rescheduled onto a newly added node group two days ago. No code or key-rotation change. Mean latency and CPU are normal. How do you triage and what's the fix?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Learn the concepts

Diagram & narrate the incident

Loading whiteboard…

Run or narrate your approach, then ask the coach.