Key Vault and secret management
Omada Identity Cloud Private uses Azure Key Vault as the single source of truth for all secrets, certificates, and sensitive configuration values. This page explains how secrets are stored, how they flow into running pods, and how to rotate them when needed.
Key Vault architecture
The deployment uses two Azure Key Vaults:
| Key Vault | Purpose |
|---|---|
Workspace Key Vault (oisaas-<workspace>) | Application secrets: SQL credentials, Application Insights keys, Entra ID service principal credentials, machine key, WinRM password. |
Shared Key Vault (privatecloud-shared) | Shared certificates: PFX certificate and password, used across workspaces. |
Secrets are mounted into pods using the Azure Key Vault Provider for Secrets Store CSI Driver. This replaces manual secret injection at deployment time – secret values live only in Key Vault and are never handled by the deployment pipeline.
How secrets reach pods
The CSI driver mounts Key Vault secrets directly into running pods and optionally syncs them to Kubernetes Secrets. The flow is:
Azure Key Vault
│
▼
CSI Secrets Store Driver (polls every 2 minutes)
│
├─► Mounted volume files inside pods
│ Updated automatically by the driver
│
└─► Synced Kubernetes Secrets
Updated automatically when mounted files change
│
└─► Pod environment variables
Set once at pod startup – not refreshed automatically
Environment variables injected from Kubernetes Secrets are set once at pod startup. Updating the secret in Key Vault does not automatically update running pods. A pod restart is required for environment variable changes to take effect.
Access control
Access to Key Vault is granted through access policies. The following identities are granted access during deployment:
| Identity | Key Vault | Permissions |
|---|---|---|
| AKS cluster identity | Workspace Key Vault | Get, List (secrets) |
| CSI Secrets Store addon identity – Workspace KV | Workspace Key Vault | Get, List (secrets) |
| CSI Secrets Store addon identity – Shared KV | Shared Key Vault | Get, List (secrets) |
| Managed Identities (all oisaas‑kl0005‑\‑mi)* | Workspace Key Vault | Get, List (one identity also has Set) |
| Azure App Service | Shared Key Vault | Full secret/key/cert lifecycle (Get, List, Update, Create, Import, Delete, Recover, Backup, Restore, Rotation Policy, Certificate Authorities, Contacts) |
All access policies are configured automatically during deployment and follow the principle of least privilege – each identity is granted only the permissions required for its function.
Certificates
Required certificates
Before deployment, the following certificates must be prepared and uploaded to Azure Key Vault. The installation script validates their presence and will stop if any are missing.
All certificates must be issued for the same domain used in your Cloud Private deployment (matching your dns_zone_name).
Primary DNS zone certificate
| Property | Details |
|---|---|
| Purpose | SSL/TLS for custom domain bindings on App Services; enables HTTPS for all external-facing endpoints |
| Format | Certificate imported directly into Key Vault (not a secret) |
| Validation | Must be in ready state with a valid secret ID |
External certificate pair
| Key Vault secret | Description |
|---|---|
aks-extCertBase64 | Base64-encoded certificate (PEM format) |
aks-extKeyBase64 | Base64-encoded private key (PEM format) |
Purpose: Secures external HTTPS endpoints. The domain must match your external DNS name.
PFX certificate bundle
| Key Vault secret | Description |
|---|---|
aks-pfxCertificate | Base64-encoded PFX/PKCS#12 bundle |
aks-pfxPassword | Password protecting the PFX file (plaintext) |
Purpose: Used by Windows container instances running Enterprise Server for IIS SSL bindings.
Obtaining certificates
Obtain valid TLS certificates for your domain from your organization's certificate authority or preferred certificate provider. All certificates must be issued for the same domain used in your Cloud Private deployment. Upload them to Azure Key Vault under the secrets defined above.
Encoding certificates for Key Vault secrets
The aks-extCertBase64, aks-extKeyBase64, and aks-pfxCertificate secrets must be Base64-encoded before uploading. The aks-pfxPassword is stored as plaintext.
The encoded values are provided in deployment-config.json under the secrets section:
Format: JSON
{
"secrets": {
"aks-extCertBase64": "<base64-encoded-external-cert>",
"aks-extKeyBase64": "<base64-encoded-external-key>",
"aks-pfxCertificate": "<base64-encoded-pfx-bundle>",
"aks-pfxPassword": "<pfx-password>"
}
}
Automatically generated secrets
The following secrets are created and populated automatically during deployment. You do not need to provide these manually.
| Secret | Purpose |
|---|---|
aks-machineKey | Shared cryptographic keys for ASP.NET encryption across all Enterprise Server pod replicas |
aks-containerRegistryDockerConfigJson | Credentials for AKS to pull container images from the container registry |
Pre-deployment checklist
Before running the installation script, verify:
- Primary DNS zone certificate exists in the shared Key Vault and is in ready state.
- External certificate and key are Base64-encoded, valid, and paired.
- PFX bundle is Base64-encoded and password-protected.
- PFX password is correct and stored in Key Vault.
- All certificate secrets are populated in
deployment-config.json. - No certificates have expired.
Secret rotation
How rotation works
The CSI driver has a built-in rotation reconciler that polls Azure Key Vault every 2 minutes. When a secret value changes in Key Vault:
- The driver detects the change within 2 minutes.
- Mounted volume files inside running pods are updated automatically.
- Synced Kubernetes Secrets are updated immediately after the volume files update.
- Pod environment variables are not updated – a pod restart is required.
How to rotate a secret
-
Update the secret value in Azure Key Vault using the Azure Portal or Azure CLI.
-
Wait up to 2 minutes for the CSI driver to detect the change and update the mounted files and Kubernetes Secrets.
-
Restart the affected pods to pick up the new value in environment variables:
Tool: kubectl
kubectl rollout restart deployment/<deployment-name> -n <namespace>
If your application reads secrets directly from the mounted file path rather than environment variables, a pod restart is not required – the updated value is available immediately after the CSI driver polls.
Prerequisites for rotation
For automatic rotation to work, the following must be in place:
| Requirement | Details |
|---|---|
| CSI Secrets Store addon enabled on AKS | Configured during deployment |
| Secret rotation enabled | Set to true with a 2-minute polling interval |
| CSI identity has Key Vault access | Get and List permissions on both Key Vaults |
| At least one pod mounting the CSI volume is running | The driver only polls Key Vault while a pod with the volume is active |
Known limitations
- No automatic pod restart. The CSI driver updates files and Kubernetes Secrets but never restarts pods. Environment variables remain stale until a manual restart.
- Rotation only works while pods are running. If all pods referencing a secret are scaled to zero, the driver stops polling. Rotation resumes when a pod mounts the volume again.
- One Key Vault per secret class. The CSI driver does not support multiple Key Vaults in a single secret class configuration. This is why the deployment uses two separate secret classes for the workspace and shared Key Vaults.