Retry amplification during vendor outage
Your task
Your account-onboarding API (Node.js) depends on a third-party KYC/identity-verification vendor over HTTPS. At 11:20 the vendor starts returning HTTP 503 for ~30% of calls and timing out on another ~20%. Your onboarding 5xx rate jumps to 50% and your own p99 doubles. The vendor's status page is green. Your retry layer is configured to retry up to 3 times on 5xx/timeout with no backoff. As you watch, the vendor's failure rate gets worse. Triage and mitigate, and explain the dynamic you're likely contributing to.
It covers
- Dependency outage
- Fail open
- Circuit breaker