Event loop synchronous crypto

On-call incidentMedium30 min

Your task

A Node.js notification service starts timing out health checks and dropping WebSocket connections under moderate load. Dashboards: event-loop lag (measured via perf_hooks) spikes from <5ms to 400ms+ and stays there; CPU on the single Node process is pinned at 100% on one core while the box has 7 idle cores; the async task queue (in-process) grows without bound. A feature shipped yesterday generates a personalized digest by doing synchronous JSON parsing + template rendering + a bcrypt-style hashing step inline in the request handler. Triage and remediate.

It covers

  • Async queue pileup
  • Cpu bound task
  • Node