Swap thrashing during RDB save

On-call incidentHard40 min

Your task

A Redis instance (used as a cache+session store) becomes briefly unresponsive, commands stall for several seconds, every hour on the hour, and clients see timeouts and a thundering retry storm. The instance holds ~12GB of data on a 16GB host with no maxmemory set and swap enabled. The stalls line up exactly with the scheduled RDB save (save 3600 ...) at the top of each hour: during the save you see memory usage briefly spike toward the host limit, swap si/so activity, and high iowait. Triage and mitigate.

It covers

  • Redis
  • Fork cow
  • Rdb snapshot