On-callMediumoc-g380

Subject Redelivery stormLevel Mid–Senior~35 minCommon in Networking & APIs interviewsIndustries Technology, Software development

Question

A `welcome-email` worker consumes a Pub/Sub subscription and calls an email provider's send API per message. At 17:00 support gets a wave of 'I received the same welcome email 3–4 times' complaints. Dashboards: the subscription's `oldest_unacked_message_age` is elevated and redelivery is high — `ack_message_count` is healthy but the *delivered/ack ratio* is well above 1; worker CPU and the email send latency both crept up after a deploy at 16:50 that added an analytics call into the send path. The ack deadline is 60s. How do you triage and mitigate?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Learn the concepts

Diagram & narrate the incident

Loading whiteboard…

Run or narrate your approach, then ask the coach.