Code RoomRetry amplification during vendor outage
HardPrep Room Coding #2418

Retry amplification during vendor outage

On-callReliability & on-callCode quality & reviewSenior–Staff~35 min

Your account-onboarding API (Node.js) depends on a third-party KYC/identity-verification vendor over HTTPS. At 11:20 the vendor starts returning HTTP 503 for ~30% of calls and timing out on another ~20%. Your onboarding 5xx rate jumps to 50% and your own p99 doubles. The vendor's status page is green. Your retry layer is configured to retry up to 3 times on 5xx/timeout with no backoff. As you watch, the vendor's failure rate gets *worse*. Triage and mitigate, and explain the dynamic you're likely contributing to.

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

0:00 of about 35 min
Which questions mattered is sealed until you submit. Telling you now would just be handing over the edge cases.