Inserts fail on month boundary

On-call incidentHard40 min

Your task

00:05 on the 1st of the month: all INSERTs into your time-partitioned events table (Postgres, declarative range partitions by month) start failing with "no partition of relation events found for row." Ingestion is down, every write for a timestamp in the new month is rejected, and the upstream queue is backing up fast. Separately, over the past few weeks a few read queries that filter on event_time have quietly gotten slower. Context: partitions were always created by hand / a one-off script when the table was set up a year ago; nobody automated future partition creation, and the cron that was supposed to pre-create next month's partition was removed in a 'cleanup' last sprint. Triage and act.

It covers

  • Time partitioning
  • Partition pruning