Code RoomNAT gateway port exhaustion from cluster scale
HardPrep Room Coding #2561

NAT gateway port exhaustion from cluster scale

On-callReliability & on-callSenior–Staff~40 min

Pods on a Kubernetes cluster intermittently see ~1s connection delays and occasional failures when calling an external third-party API, starting after the cluster autoscaled from 30 to 120 nodes for a campaign. The failures correlate with a single NAT gateway used for egress. The third-party API is a single VIP. Cloud NAT metrics show 'allocated ports' near the per-instance maximum and a rising 'dropped/sNAT-port-exhaustion' counter. App-side fd counts and pool sizes look healthy on each pod. Triage and mitigate.

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

0:00 of about 40 min
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.