Code Room
On-callHardoc-g396
Subject Message lossLevel Senior–Staff~40 minCommon in Reliability & on-call interviewsIndustries Technology

Question

A `clickstream` Kafka consumer group fell behind during a long outage of its downstream warehouse (the consumer was effectively paused for ~14 hours while the warehouse was down). The topic `clickstream` is configured with `retention.ms` = 12 hours (`cleanup.policy=delete`). When the consumer resumes at 08:00, it starts processing again with no errors and no lag alerts clearing the way you'd expect — and reconciliation later finds a ~2-hour window of events that were NEVER delivered to the warehouse, even though the consumer 'caught up.' Dashboards: during resume, the consumer logged `OffsetOutOfRangeException` on several partitions, and `auto.offset.reset=latest`. How do you triage and mitigate?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Diagram & narrate the incident
Loading whiteboard…
Run or narrate your approach, then ask the coach.