Postgres replica CPU spike from unoptimized query

On-call incidentMedium30 min

Your task

At 12:40 your reporting service's Postgres read replica spikes to 100% CPU and stays there; every query on that replica times out. pg_stat_activity shows one query running for 14 minutes that the app never normally runs that long: a user-built ad-hoc report joining three large tables with a LIKE '%...%' free-text filter and no LIMIT, generated by a new self-serve report builder shipped this morning. The same query keeps reappearing because the user keeps hitting 'retry' after each timeout. How do you triage and mitigate?

It covers

  • Regex
  • Cartesian
  • Cpu saturation