Provider rate limit on fan-out

On-call incidentMedium30 min

Your task

Your notification service sends SMS via a third-party provider (Twilio-style). After a product launch, users report missing texts. Your logs show a wall of HTTP 429 Too Many Requests from the provider with a Retry-After header, and your provider dashboard shows you've hit your account's per-second message quota. Your service fans out one API call per recipient with no client-side limiting, and you just retry immediately on 429. The launch announcement targeted 500k users in a 10-minute window. How do you triage and get messages flowing without burning more quota?

It covers

  • Api rate limit
  • Third party
  • Fan out