Code RoomLarge message OOM crash loop
MediumPrep Room Coding #2502

Large message OOM crash loop

On-callReliability & on-callMid–Senior~30 min

A document-processing worker pool consuming a RabbitMQ queue enters a crash loop at 16:20. Dashboards: each worker pod OOM-kills and restarts every ~90s; the queue depth is rising; on restart, workers immediately reconnect, the same message is redelivered (it was un-acked), and they crash again. Memory profiler from a surviving pod shows a single message being loaded into memory expands to >6GB. Context: a customer just uploaded a deeply-nested 800MB JSON that the worker fully parses into objects before processing. There's no DLQ configured. How do you triage, break the crash loop, and process the rest of the queue?

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

0:00 of about 30 min
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.