Disk full on primary

On-call incidentHard40 min

Your task

Postgres primary. Disk usage on the data volume climbed steadily over 2 days and just hit 100%; the DB is now refusing writes / crashing on could not write to file ... No space left on device. The pg_wal directory is huge. There was no traffic spike. Two weeks ago a team set up a logical-replication subscriber for a new data pipeline; that subscriber has been down for days. Triage, get writes back safely, and prevent recurrence.

It covers

  • Wal disk full
  • Replication slot
  • Wal retention