Secrets distribution at scale
Design a secrets-distribution system for ~8,000 microservice instances that need database credentials, API keys, and TLS certs. Requirements: no long-lived secret ever sits in an env var or image; credentials rotate automatically; a leaked credential is useless within minutes; and a full audit trail of who/what accessed which secret. Static-secret reads peak at 30K/sec. Discuss dynamic (short-lived, on-demand) secrets vs rotating static secrets, how a service authenticates to the secrets service in the first place (the bootstrapping/'secret zero' problem), and the trade-off when the secrets service is down.
Clarify scale and constraints first. Propose a clean component breakdown, then go deep on the hard parts (data model, bottlenecks, consistency, failure modes) and name the trade-offs you are making.