Scale-in terminates in-flight requests
Your task
After an evening traffic peak passes, your autoscaler scales the fleet in to save cost. Immediately you get a wave of errors: in-flight requests fail with connection resets, a batch of long-running export jobs vanish and never complete, and customers see 'export failed'. The dashboards show instances being terminated within seconds of the scale-in decision, mid-request. The service handles requests up to 3 minutes long and runs background jobs in-process. CPU/memory are fine, the errors are purely from instances disappearing. How do you triage and make scale-in safe?
It covers
- Scale in
- Termination
- Stateful