JWT signing key rotation after leak
An incident reveals your JWT signing key (HS256 shared secret) may have leaked via a verbose error response that a customer screenshotted in a support ticket weeks ago. Tokens are valid for 24h and there are ~3M active sessions across web + mobile. If the secret is compromised, an attacker can forge tokens for any user, including admins. You must rotate the signing key, but a naive rotation invalidates all 3M sessions at once (mass logout, mobile re-auth storm) and the secret is also used by 5 backend services to verify tokens. How do you triage and rotate without a forgery window or a self-inflicted outage?
Stop the bleeding first (mitigate), then form hypotheses from real signals. Separate root cause from symptom, communicate status as you go, and close with what prevents a repeat.