System Design Interview: A Secrets and Key Management Service
4 views
"Design a service that stores API keys, database passwords, and encryption keys for thousands of applications" (think HashiCorp Vault or a cloud KMS) is a security-focused design question asked at infrastructure, cloud, and fintech companies. It tests encryption design, access control, availability, and auditing.
1. Requirements
Functional: applications store and retrieve secrets by name; secrets have versions and can be rotated; operators define access policies; the service can encrypt and decrypt data with managed keys without exposing the keys; every access is audited.
Non-functional: secrets must never be stored or logged in plaintext; very high availability, since applications can't start without their secrets; low latency for reads; strong consistency for policy changes and revocations.
2. Envelope encryption
Explain the key hierarchy:
- A root key (master key) is kept in hardware security modules (HSMs) and never leaves them.
- The root key encrypts key-encryption keys, which encrypt per-secret data encryption keys (DEKs).
- Each secret is encrypted with its own DEK; the encrypted DEK is stored alongside the ciphertext.
To read a secret, the service asks the HSM layer to decrypt the DEK, decrypts the secret in memory, returns it over TLS, and discards the plaintext key. Rotating the root key only requires re-encrypting the key-encryption keys, not every secret.
3. Authentication and authorization
Applications authenticate with workload identity rather than another static password: cloud instance identity, Kubernetes service account tokens, or mutual TLS certificates. Policies map identities to paths and actions ("payments-service can read secrets/payments/*"). Evaluate policies on every request and cache them briefly, invalidating on change.
4. Storage
Store encrypted secrets, versions, and metadata in a replicated, strongly consistent store (for example a Raft-based cluster or a managed database with synchronous replication). Since data is encrypted before storage, the storage layer never sees plaintext.
5. Availability
Run multiple replicas across availability zones with leader election. Clients cache secrets in memory with a short time-to-live so a brief outage doesn't take down applications. Discuss the trade-off: longer caching improves resilience but delays revocation.
6. Rotation and dynamic secrets
Support versioning so applications can roll to a new secret without downtime. Go further with dynamic secrets: the service creates short-lived database credentials on request and revokes them automatically when they expire, which limits damage from leaks.
7. Auditing
Write every read, write, and policy change to an append-only audit log shipped to a separate system, with requester identity, secret path (never the value), and timestamp. Alert on unusual access patterns.
8. Disaster recovery
Back up encrypted data and plan HSM key backup carefully; losing the root key means losing every secret.
Staying structured in the round
Security designs have many layers to cover. Linkjob AI (https://linkjob.ai) gives real-time help during live job interviews and technical interviews, reminding you of areas such as rotation or auditing while you lead your own design.