Serverless cold start wave
Your task
Your image-resize endpoint runs on a serverless platform that scales to zero when idle and scales out one new instance per unit of concurrency. A marketing email goes out at 10:00 and within seconds thousands of users click through. Almost every one of those first requests takes 3–5s instead of the usual 150ms, and a chunk of them time out at the CDN. After ~30 seconds latency returns to normal. Logs show a flood of init / coldstart markers at 10:00:0x. There was no deploy. How do you triage and prevent the next email from doing this?
It covers
- Scale to zero
- Concurrency burst
- Init latency