Cap concurrent requests per API key across instances

Vibe codingHard30 minStripe style

Your task

Your TypeScript API runs on Express across 8 instances that share Redis. Some endpoints build reports that take up to 2 minutes, and last week one customer's script started 300 at once and starved everyone else.

You want to cap how many requests each API key has running at the same time, across all instances. An agent will build it as Express middleware.

Write the spec you would hand the agent.

What you know:

  • Node 20, Express 4, ioredis and Redis 7
  • the limit is 10 requests at once on most plans and 25 on enterprise
  • the load balancer cuts any request at 180 seconds
  • instances restart on every deploy, a few times a day, and now and then one crashes

Then, in a few sentences: why is this harder than a requests-per-minute limit?