Container pull latency on scale-from-zero

On-call incidentMedium35 minGoogle style

Your task

Your service runs on a platform that scales to zero when idle (Knative/Cloud Run style). A monitoring synthetic and the first real users after an idle period get 10–20 second responses or timeouts, while warm responses are ~150ms. The platform logs show the request waiting on a new pod being created: a large container image is pulled, the runtime starts, and only then does the app boot and connect to its dependencies. Traffic is spiky, long idle gaps then sudden bursts. How do you triage and cut the first-request latency?

It covers

  • Scale from zero
  • Container
  • Image pull