Version: Omada Identity Cloud Private

Post-deployment health check

The health check script (health_check.sh) validates that a deployed Omada Identity Cloud Private environment is running correctly. It checks endpoint availability, pod readiness, component versions, error logs, and Service Bus dead letters, and produces a summary report suitable for pre-handover validation or routine operational checks.

Prerequisites

Before running the script, ensure the following:

Azure CLI is installed and authenticated (az login) against the correct subscription.
jq and curl are available on the machine running the script.
The executing identity has the following RBAC permissions on the environment resource group:

Permission	Purpose
`Microsoft.ContainerService/managedClusters/read`	AKS – read cluster
`Microsoft.ContainerService/managedClusters/listClusterAdminCredential/action`	AKS – list admin credentials
`Microsoft.Web/sites/read`	App Service – read site
`Microsoft.Web/sites/config/read`	App Service – read config
`Microsoft.Web/sites/config/list/action`	App Service – list app settings
`Microsoft.OperationalInsights/workspaces/read`	Log Analytics – read workspace
`Microsoft.OperationalInsights/workspaces/query/*/read`	Log Analytics – query
`Microsoft.ServiceBus/namespaces/read`	Service Bus – list namespaces
`Microsoft.ServiceBus/namespaces/topics/read`	Service Bus – list topics
`Microsoft.ServiceBus/namespaces/topics/subscriptions/read`	Service Bus – list subscriptions

note

Some of these permissions are not included in the built-in Reader role. Assign a custom role covering all 10 actions, or use Contributor / Owner. RBAC changes can take 5–10 minutes to propagate after assignment.

Running the script

From the repository root, run:

Tool: Bash shell

./scripts/health_check.sh --global-prefix <global-prefix> [--log-window-minutes <n>]

Flag	Required	Description
`--global-prefix`	Yes	Environment global prefix (for example, `oisaas-ab001`)
`--log-window-minutes`	No	Time window for error log check in minutes (default: `30`)
`--tfvars-path`	No	Path to `variables.auto.tfvars.json` (default: `iac/infrastructure/iac/variables.auto.tfvars.json`)
`--apps-config`	No	Path to `health_check_apps.json` (default: `scripts/health_check_apps.json`)
`--dns-zone`	No	DNS zone for the Enterprise Server (AKS) endpoint. Auto-detected from `config.artifact_source.dns_zone_name` in `--tfvars-path` if not provided. Pass explicitly to override.
`--retries`	No	Extra retry attempts for transient failures (default: `2`)
`--retry-delay`	No	Seconds between retry attempts (default: `2`)
`--manifest`	No	Path to a manifest JSON to use as the expected-version source (enables manifest mode)
`--use-manifest`	No	Manifest mode: use a local `manifest.json`, or fetch `<global-prefix>.json` from blob storage if no local file exists

Exit codes:

0 – all checks passed, or only warnings
1 – one or more FAIL results

What the script checks

The script runs five checks in sequence.

Check 1: Endpoint ping

Verifies that all services respond on their expected endpoints. All requests run in parallel – the total time equals the slowest single endpoint (15-second timeout).

Result	HTTP response
PASS	`200`, `301`, `302`, `401`
WARN	`404` (endpoint not found), `503` (cold start)
FAIL	`000` (no connection / timeout), `5xx`

Web apps are checked at /swagger/index.html. Function apps are checked at /api/Heartbeat – a 404 response is treated as WARN rather than FAIL, since not all function apps expose an HTTP trigger.

Check 2: AKS pod readiness

Checks that each AKS deployment has at least one ready pod. The following deployments are checked:

Deployment	Service
`<alias>-es`	Enterprise Server
`<alias>-rope`	Role and Policy Engine
`<alias>-timer`	Timer
`<alias>-cagops`	CAG Operations

If kubectl is unavailable or the AKS cluster is unreachable, this check is skipped with WARN.

Check 3: Component version verification

Confirms that all running components match the versions declared in variables.auto.tfvars.json. Checks both App Service / Function App blob URLs (WEBSITE_RUN_FROM_PACKAGE) and AKS container image tags. All app settings calls run in parallel.

By default, expected versions are read from variables.auto.tfvars.json (config mode); with --manifest or --use-manifest, they are read from a release manifest instead.

A component running a different version than expected is reported as FAIL. A component with WEBSITE_RUN_FROM_PACKAGE not set is also reported as FAIL.

Check 4: Error log report

Queries the Log Analytics workspace (<global-prefix>-law) for errors in the OIS custom log table (OIS_CL) within the configured time window. Errors at level 4 (Error) or 5 (Fatal) are reported as WARN – not FAIL, since isolated log errors do not necessarily block handover.

note

If Log Analytics is not enabled for the environment (for example, in some development workspaces), this check is skipped and reported as N/A.

Check 5: Service Bus dead letters

Checks all Service Bus namespaces in the resource group for dead-lettered messages. Dead letters indicate processing failures that were not retried successfully.

Dead letter count = 0 → PASS
Dead letter count > 0 → WARN with per-topic/subscription breakdown

Namespace and topic checks run in parallel at two levels – the total time equals the slowest single namespace.

Interpreting results

Each check produces a line per resource with a PASS, WARN, or FAIL label, followed by a summary; for example:

======================================================
 Omada Private Cloud – Environment Health Check
 Workspace : oisaas-ab001
 Date/Time : 2026-05-26 14:32 UTC
 Version   : 1.0.13
======================================================

[1/5] ENDPOINT PING
  PASS  oisaas-ab001-wa-copsapi   (200)
  FAIL  oisaas-ab001-wa-ods       (000 - no connection / timeout)
  ...
  elapsed: 18s

[2/5] AKS POD READINESS
  PASS  oisaas-es     1 pod(s) ready
  FAIL  oisaas-rope   no ready pods
  ...

[3/5] BLOB VERSION CHECK
  PASS  oisaas-ab001-wa-copsapi   418163_06fa5d…
  FAIL  oisaas-ab001-wa-ods       running: 414000_abcd…  expected: 415909_6fcbb8…
  ...

[4/5] ERROR LOGS (last 30m)
  WARN  OIS_CL – 2 component(s) with errors:
          Component=ImportJob  Level_d=4  Count=3
  ...

[5/5] SERVICE BUS DEAD LETTERS
  PASS  oisaas-ab001-en-bus  0 dead letters
  WARN  oisaas-ab001-ci-bus  7 dead letters
          topic-a/sub-1       7 dead letters
  ...

======================================================
 SUMMARY: 28 passed  |  1 failed  |  2 warned
 RESULT : FAIL – environment not ready for handover
======================================================

The final result is:

PASS – all checks passed or warned only; environment is ready for handover.
FAIL – one or more checks failed; investigate before handover.

Known limitations

Permission pre-check is informational only. WARN results in the permissions pre-check do not abort the run – they indicate which downstream checks are likely to fail and why. notActions (deny overrides) are not evaluated.
AKS checks require kubectl. If kubectl is absent or the cluster is unreachable, check 2 is skipped with WARN, and check 3's AKS container-tag check is skipped with WARN as well (its blob-version checks still run).
Log Analytics must be enabled. Check 4 is skipped with N/A in environments where Log Analytics is not configured.
Timeout flakiness. Occasional FAIL results on timeout-sensitive checks may resolve on a second run.
Naming convention dependency. The script derives resource names from the global prefix. Customized resource names require updating scripts/health_check_apps.json.

tip

If the health check reports failures or warnings and you need to troubleshoot further, see Collecting debug logs to gather diagnostic data for analysis or to send to Omada Support.

Prerequisites​

Running the script​

What the script checks​

Check 1: Endpoint ping​

Check 2: AKS pod readiness​

Check 3: Component version verification​

Check 4: Error log report​

Check 5: Service Bus dead letters​

Interpreting results​

Known limitations​