Gateway request queue blocking
Your task
An API gateway (single shared worker pool / request queue) fronts both fast lightweight reads (~5ms) and a handful of expensive report-export endpoints (~8s each). At 15:00 a customer scripts a burst of report-export calls. Suddenly all traffic through the gateway, including the 5ms health checks and reads, sees latency jump to multiple seconds and some time out. Export volume is small in count but each is long-running. Triage and explain the blocking mechanism, then mitigate.
It covers
- Http1 pipelining
- Single queue
- Priority