AI feature far over its daily budget

On-call incidentHard35 minOpenAI and Anthropic style

Your task

At 09:14 your provider billing alert fires: yesterday your AI feature spent $41,000 against a $600 daily budget, and today is tracking higher. The feature is metered per user, nothing has been deployed in six days, error rates are normal and users are not complaining. Walk through your first 30 minutes. Cover how you stop the bleeding without taking the product down, how you tell a runaway retry loop from an abuse pattern from a genuine traffic spike, and what you put in place before you go back to sleep.

It covers

  • Finops
  • Metered spend
  • Quota
  • Blast radius