On-callHardoc-g423

Subject Connection pool exhaustionLevel Senior–Staff~35 minCommon in Reliability & on-call interviewsIndustries Technology

Question

A managed-Postgres failover just promoted a standby (took ~25s). Right after the new primary came up, your API started returning 500s in waves and Postgres logs are full of `FATAL: sorry, too many clients already`. You run PgBouncer in transaction mode in front of ~40 app instances. `pg_stat_activity` on the new primary is pinned at `max_connections`. The failover itself completed cleanly; the database is up. Connections keep spiking, dropping, and spiking again. Triage and mitigate.

What a strong answer looks like

Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.

Learn the concepts

Diagram & narrate the incident

Loading whiteboard…

Run or narrate your approach, then ask the coach.