New pods cannot log in to the database

At 09:00 the platform team starts a routine node upgrade, which drains nodes and reschedules pods. By 09:12, 4 of the 10 billing API pods are in CrashLoopBackOff, each logging at startup:

FATAL: password authentication failed for user "billing_app"

The 6 older pods are still serving on their existing database connections. The error rate is 0.4%, but latency is creeping up as those 6 pods carry all the traffic. Nobody has changed the billing-db secret in 11 months, and the password in it matches the one in the team's vault. The database team's wiki says their provisioning script creates service roles with a VALID UNTIL date one year ahead. The billing service launched a year ago this week. How do you triage and mitigate?