Code Room
On-callMediumoc-g554
Subject On callLevel Mid–Senior~35 minCommon in Reliability & on-call · Code quality & review interviewsIndustries Technology

Question

At 00:00:30 UTC, right after midnight, your service-to-service calls to the internal `inventory` API begin failing with `certificate has expired` / `tls: failed to verify certificate`. It's all-or-nothing for that one dependency — every call fails — and it started exactly at midnight. Other dependencies are fine. The inventory team made no deploy; nothing changed on either side in the last 12 hours. The inventory API's TLS cert was issued by your internal CA. Browsers hitting inventory's public edge (which uses a different, Let's Encrypt cert) work fine. Pages are flooding in for every service that calls inventory. Triage this and lay out the immediate fix and the prevention.

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Diagram & narrate the incident
Loading whiteboard…
Run or narrate your approach, then ask the coach.