Connection pool exhaustion after autoscale

On-call incidentHard35 min

Your task

A fleet of 60 stateless API pods (Node.js) each holds a Postgres pool of 20 connections. After an autoscaling event scaled the fleet from 30 to 60 pods during a traffic spike, the database starts rejecting new connections with 'FATAL: sorry, too many clients already' (Postgres max_connections = 500). API error rate spikes. The DBA refuses to raise max_connections further because each Postgres backend costs ~10MB and CPU context-switching is already high. Triage and propose both the immediate fix and the right architecture.

It covers

  • Connection pool
  • Pgbouncer
  • Max connections