Architecture Azure

Enterprise Pattern: Binding a Cross-Subscription Key Vault Certificate to Application Gateway

In a properly segmented Azure landing zone, certificates are not scattered — they are centralized. In one enterprise platform I ran, the wildcard cert *.contoso.com lived in a Key Vault in the Identity (or Management) subscription, the Application Gateway that terminated TLS for every public workload lived in the Connectivity subscription, and the dozen-odd apps behind it sat in their own spoke subscriptions. Clean separation of duties, one place to rotate and audit certs — exactly what the Cloud Adoption Framework asks for. Until you open the portal to attach that cert to the gateway’s HTTPS listener and discover the Key Vault picker only lists vaults in the gateway’s own subscription. The central, cross-subscription certificate simply is not selectable. Teams hit this wall, conclude “cross-subscription certs aren’t supported,” and scatter copies of the wildcard into per-subscription vaults — quietly destroying the governance they so carefully designed.

That conclusion is wrong. This is a portal limitation, not a platform one. Application Gateway pulls a certificate from Key Vault as a user-assigned managed identity holding a single secret-read permission, and that works perfectly across subscription boundaries inside one Microsoft Entra tenant — because RBAC scope does not care about subscription lines. You wire it through the CLI, ARM, Bicep, or Terraform, and it serves the central cert and auto-rotates when that cert is renewed in the other subscription. This article is the whole pattern, end to end, at the depth a senior cloud architect actually needs: every moving part, every option, every way it fails in production, and the exact az, Bicep and Terraform to build it and confirm each failure.

By the end you will understand the integration as a system, not a recipe: why it must be a user-assigned identity, why you bind the /secrets/ URL and not the /certificates/ URL, why an unversioned secret ID is the difference between auto-rotation and a frozen-forever cert, how the gateway’s roughly-every-few-hours poll interacts with renewal timing, how a Private Endpoint and the Key Vault firewall change the network path, and how soft-delete, purge protection, and a missed notBefore quietly defeat the design. You will be able to walk into an incident where the gateway serves an expired cert despite a freshly renewed vault, and name the cause in ninety seconds.

What problem this solves

Centralizing TLS material in one Key Vault is the right architectural call. One vault is one place to govern: one rotation runbook, one audit log, one purge-protection setting, one network boundary. Scatter the same wildcard into ten spoke vaults and you have ten rotation jobs that drift out of sync, ten audit surfaces, and ten chances for a stale or unprotected copy of your most sensitive key material. The first time a wildcard renews and only seven of the ten vaults get the new version, you have a fleet of gateways serving a mix of old and new certs and no single source of truth.

So the central vault is correct — and the moment you adopt it, you collide with two friction points that this article exists to remove. First, the portal gap: the Application Gateway listener UI cannot see a Key Vault in another subscription, so the naive path (portal → listener → pick certificate) is a dead end and people wrongly conclude the topology is unsupported. Second, and far more dangerous, the silent auto-rotation trap: even when you wire it correctly through code, one ambiguous choice — binding the versioned secret URL instead of the unversioned one — pins the gateway to a single cert version forever. The renewal lands under a new version, the old one expires, and the gateway keeps serving the dead one. The cert is renewed; the gateway is broken; nothing in the portal screams about it until users get certificate-expired errors in the browser.

Who hits this: every platform team running Enterprise-Scale / Cloud Adoption Framework landing zones with a dedicated Connectivity subscription and a central secrets/identity subscription. It bites hardest the first time you stand up a hub Application Gateway, and again — months later, in production — the first time the central wildcard rotates and a versioned binding refuses to follow it. The same identity-plus-secret-read pattern generalizes to Front Door, API Management, and App Service pulling central certs across subscriptions, so getting it right once pays off across the estate.

Learning objectives

By the end of this article you can:

Prerequisites & where this fits

You should already be comfortable with the building blocks this pattern composes. You know Application Gateway v2 (Standard_v2 / WAF_v2) — a regional Layer-7 load balancer that terminates TLS, routes by host/path, and runs a WAF — lives in a dedicated subnet in a VNet. You know a Key Vault certificate is exposed three ways: a certificate (public cert + policy), a key (private key, non-exportable by default), and a secret (the full PFX/PEM, base64). You know managed identity (system- vs user-assigned) and Azure RBAC (roles, scope, assignments; scope can be MG, subscription, RG, or a single resource). You can run az in Cloud Shell, read JSON, and have skimmed Bicep or Terraform.

This sits in the Networking / Identity / Platform-Engineering intersection of the landing-zone story. Upstream are the segmentation decisions: the Enterprise-scale connectivity subscription and hub design that puts the gateway in Connectivity, and the hub-and-spoke vs Virtual WAN topology choice that gives it a route to a Private Endpoint. The identity mechanics come from Entra managed identities: user-assigned, federated credentials and RBAC and the system-assigned vs user-assigned patterns decision. The permission model is the Key Vault RBAC vs access policies split; the certificate object is covered in Azure Key Vault: secrets, keys and certificates; and the gateway’s broader TLS story lives in Application Gateway with WAF, mTLS and end-to-end TLS. Read this one as the cross-subscription certificate-binding deep dive those articles point at.

A quick map of who owns what across the boundary, so during a build or an incident you call the right team fast:

Layer What lives here Which subscription Who usually owns it What it can break in this pattern
Central Key Vault The wildcard cert object, soft-delete, purge protection, firewall Identity / Management Security / Platform Soft-delete collision, firewall block, secret disabled
RBAC role assignment Key Vault Secrets User granting the UAMI GET on secrets Scoped to the vault (Identity sub) Security / Platform Missing assignment → cert never loads
User-assigned identity The principal the gateway authenticates as Connectivity Network / Platform Wrong identity attached, or none
Application Gateway The listener, the SSL cert reference, the WAF Connectivity Network team Versioned binding, SKU mismatch, Unknown cert state
VNet / subnet / route The path from the gateway to the vault’s Private Endpoint Connectivity (+ spoke) Network team No route to PE, DNS not resolving private IP
Private DNS zone privatelink.vaultcore.azure.net resolution Connectivity / hub Network team Gateway resolves public IP, firewall blocks it
Spoke workloads The apps the gateway routes to Spoke subscriptions App teams Out of scope for cert binding; backend health only

Core concepts

Six mental models make every later step and every failure obvious. Internalize these and the CLI is just typing.

A Key Vault certificate is read through its secret representation. When you create or import a certificate into Key Vault, the platform stores three linked addressable objects under the same name: a certificate object (/certificates/<name> — the X.509 public cert and the issuance policy), a key object (/keys/<name> — the private key, generated as non-exportable unless the policy says otherwise), and a secret object (/secrets/<name> — the entire certificate including the private key, serialized as a base64 PFX or as PEM depending on the content type). Application Gateway needs the whole thing — public cert and private key — to terminate TLS, so it reads the secret, not the certificate object. That is why every URL and every permission in this pattern says secrets, and why a role that grants certificate-read but not secret-read still fails. This trips up almost everyone the first time.

The gateway authenticates as a user-assigned managed identity — and it must be user-assigned. Application Gateway’s Key Vault integration is implemented against a user-assigned managed identity (UAMI) attached to the gateway; a system-assigned identity is not supported for this integration, and wiring the listener with only a system-assigned identity leaves the cert reference unresolved. The UAMI is a standalone resource with its own principalId (the Entra service-principal object ID). You create it once, attach it to the gateway, and grant it the vault permission — and because it is a first-class tenant principal, the assignment works regardless of which subscription the vault sits in.

Subscriptions are a billing and management boundary, not an identity boundary. All subscriptions in one Entra tenant share the same directory. An RBAC assignment binds a principal to a role at a scope; here the scope is the central vault’s resource ID (encoding its subscription) and the principal is the UAMI (in another subscription). Entra resolves the principal globally; ARM evaluates the assignment at the vault’s scope; the subscription difference is irrelevant to the decision. This is the single fact that makes the pattern work — and why the portal’s per-subscription picker is a UI shortcoming, not a platform rule. (Moving resources between subscriptions is separate — see moving resources across groups and subscriptions — but you rarely need to.)

Unversioned secret ID = auto-rotation; versioned = frozen. A Key Vault secret has two URL forms: the versioned ID pins an exact version (…/secrets/wildcard-contoso/8a1b2c3d...); the unversioned ID omits the trailing version (…/secrets/wildcard-contoso). Application Gateway polls Key Vault roughly every few hours and re-fetches the secret behind whatever URL you bound. Bind the unversioned URL and the poll always resolves to the current version, so a renewal is picked up automatically — the entire value of central rotation. Bind the versioned URL and the poll fetches that exact, now-frozen version forever; the renewal lands as a new version the gateway never looks at, the pinned one expires, and the gateway serves a dead cert. The cruel detail: az keyvault secret show --query id returns the versioned URL, so the wrong choice is the default if you copy-paste blindly. Strip the trailing segment yourself.

RBAC reaches; the network connects. Two independent gates sit between the gateway and the cert. RBAC (or an access policy) decides whether the identity is allowed to read the secret — an Entra/ARM authorization decision. The network decides whether the gateway can reach the vault’s data-plane endpoint — a routing-and-firewall decision. A vault firewalled to “selected networks” refuses a connection even from a perfectly-authorized identity arriving from an untrusted IP. The classic “authorized but unreachable” symptom is a cert in state Unknown — authorization passed, but the fetch can’t complete. You must solve both: grant the role and open a path (Private Endpoint or trusted-services).

Soft-delete and purge protection are always on — and they bite during teardown and re-create. Modern Key Vaults have soft-delete permanently enabled (retention 7–90 days, default 90), and many enterprise vaults additionally have purge protection on (blocking early purge until retention elapses). This matters here because cert and vault names are reserved while soft-deleted: delete a vault or cert and try to recreate it with the same name, and you collide with the soft-deleted shadow — Conflict/ConflictError — until you recover or (if allowed) purge it. Know where the recovery commands are before you need them; Key Vault 403 / soft-delete recovery covers them in depth.

The vocabulary in one table

Before the deep sections, pin every term down. The glossary at the end repeats these for lookup; this is the mental model side by side:

Concept One-line definition Where it lives Why it matters to this pattern
Central Key Vault The single vault holding the wildcard cert Identity / Management sub One place to rotate/audit; the source of truth
Certificate object /certificates/<name> — public cert + policy Key Vault NOT what the gateway reads
Secret object /secrets/<name> — full cert incl. private key (PFX/PEM) Key Vault What the gateway actually fetches
User-assigned identity (UAMI) Standalone principal the gateway authenticates as Connectivity sub Required (system-assigned unsupported here)
principalId The UAMI’s Entra service-principal object ID The UAMI The assignee of the role grant
Key Vault Secrets User Built-in role granting GET on secrets Role definition (tenant) The least-privilege grant the UAMI needs
Unversioned secret ID …/secrets/<name> (no version) The listener’s sslCertificate Enables auto-rotation
Versioned secret ID …/secrets/<name>/<guid> A frozen binding Defeats auto-rotation — the classic bug
Poll interval Gateway re-fetches the secret every few hours Platform behaviour Why rotation lags by hours, not instant
Private Endpoint (KV) A private IP for the vault in a subnet Connectivity / hub VNet The network path to a firewalled vault
KV firewall “Selected networks” + trusted-services toggle Key Vault networking Blocks the fetch → cert state Unknown
Soft-delete / purge protection Deleted vaults/certs are recoverable, names reserved Key Vault Re-create collisions; recovery before purge
Cert state Unknown Gateway can authenticate but not fetch Gateway SSL cert status The “reaches but can’t connect” tell

The certificate object: why /secrets/, not /certificates/

This concept, when missed, sends people down a multi-hour wrong path, so it gets its own section. When you put a certificate into Key Vault — generated there or imported as PFX — Key Vault materializes one logical certificate as three addressable objects sharing the name. The gateway terminates TLS, which requires the private key, so it reads the object that holds the private key in a loadable form: the secret.

Key Vault object URL form Contains Who reads it for TLS term Permission needed
Certificate https://<vault>/certificates/<name>[/<ver>] X.509 public cert + issuance policy Tools inspecting/renewing the cert Certificates GET (e.g. Key Vault Certificates Officer/User)
Key https://<vault>/keys/<name>[/<ver>] The private key (non-exportable by default) Crypto operations (sign/decrypt) Keys permissions
Secret https://<vault>/secrets/<name>[/<ver>] Full cert incl. private key (base64 PFX or PEM) Application Gateway, App Service, Front Door Secrets GET (Key Vault Secrets User)

Two practical consequences fall out of this:

First, the role you grant is a secrets role. Key Vault Certificates User is not enough — it grants certificate-object reads, not secret reads, so the gateway’s fetch of /secrets/... returns Forbidden. The correct built-in role is Key Vault Secrets User (4633458b-17de-408a-b874-0445c86b69e6), granting secrets/getSecret/action and readMetadata. On the legacy access-policy model, the equivalent is secret permission: get (you do not need list — don’t grant it).

Second, the URL you bind is a secrets URL. A common mistake is to copy the certificate identifier (/certificates/...) and hand that to --key-vault-secret-id; the gateway then fails to load it. Always derive the binding from the secret identifier. The relationship between the cert name and the three URLs:

You have / you want Command Returns Use for the gateway?
The certificate identifier az keyvault certificate show --vault-name kv -n wildcard --query id …/certificates/wildcard/<ver> No
The secret identifier (versioned) az keyvault secret show --vault-name kv -n wildcard --query id …/secrets/wildcard/<ver> No — strip the version first
The secret identifier (unversioned) ${SID%/*} on the above …/secrets/wildcard Yes
The KID (key id) az keyvault certificate show … --query kid …/keys/wildcard/<ver> No

A note on content type: a cert with content type application/x-pkcs12 yields a base64 PFX in the secret; application/x-pem-file yields PEM. Application Gateway accepts the secret as the platform stores it — you convert nothing. Do not enable secret export or copy the PFX out as a workaround; the point is that only the gateway’s identity ever touches the private key, in place.

The certificate-vs-secret content-type and what consumes which:

Content type Secret returns Created by Consumed by Notes
application/x-pkcs12 Base64-encoded PFX (cert + private key) KV cert generate/import (PFX) App Gateway, App Service, Front Door The common case for AppGW
application/x-pem-file PEM (cert + key concatenated) KV cert generate/import (PEM) Same consumers Equivalent; just a different serialization
Imported cert without key Public cert only (no key) Importing a .cer Trust stores / client validation Won’t terminate TLS — no private key

The relationship between the gateway’s listener and the cert objects, end to end:

Listener element Points at Which resolves to Auto-rotates?
httpListener.protocol = Https An sslCertificate on the gateway
sslCertificate.keyVaultSecretId The KV secret id The latest version (if unversioned) Yes (unversioned) / No (versioned)
The fetched secret The PFX/PEM Cert + private key the gateway loads Tracks the bound id
The gateway’s UAMI Its principalId The role assignment on the vault N/A (auth, not rotation)

The identity model: user-assigned, and why

Application Gateway reads Key Vault material as a managed identity, and for this integration it must be a user-assigned one — worth being precise about why, because “just use user-assigned” without the reasoning leads people to attach the wrong thing.

A system-assigned identity is tied 1:1 to a single resource’s lifecycle. The gateway can have one for other purposes, but the Key Vault certificate integration specifically requires a user-assigned identity — a standalone Microsoft.ManagedIdentity/userAssignedIdentities resource you create independently, attach to multiple gateways, and rotate without destroying the principal. That independence is operationally nice: grant the vault permission to one stable principal, and every gateway that serves the central cert shares it.

Identity property System-assigned User-assigned (UAMI) Relevance here
Lifecycle Born/dies with the resource Independent resource UAMI survives gateway re-creation
Supported for AppGW→KV cert No (not for this integration) Yes (required) Must use UAMI
Reusable across resources No (1:1) Yes (1:many) One grant serves many gateways
Has stable principalId Yes, but ephemeral Yes, stable Stable assignee for RBAC
Created with identity { type: 'SystemAssigned' } az identity create Separate az identity create step
Granted the vault role to Its principalId Its principalId Same role either way (Key Vault Secrets User)

The mechanics: you create the UAMI in the Connectivity subscription, capture its resource ID (/subscriptions/<conn>/resourceGroups/rg-connectivity/providers/Microsoft.ManagedIdentity/userAssignedIdentities/id-appgw-kv) and its principalId (the Entra object ID). The resource ID is what you attach to the gateway; the principalId is what you grant the role to. Keep them straight — attaching the wrong ID or granting the role to the gateway’s own resource ID instead of the identity’s principalId are both common slip-ups.

The three UAMI identifiers and where each one goes — getting these crossed is the second-most-common wiring mistake after the versioned binding:

Identifier az identity show query Looks like You use it to…
Resource ID --query id /subscriptions/.../userAssignedIdentities/id-appgw-kv Attach the identity to the gateway
principalId --query principalId A GUID (Entra object ID) Grant the role assignment (the assignee)
clientId --query clientId A GUID (app/client ID) Token requests in app code (not needed here)
# In the Connectivity subscription
az account set --subscription "$CONNECTIVITY_SUB"
az identity create -g rg-connectivity -n id-appgw-kv -l centralindia -o table

UAMI_ID=$(az identity show -g rg-connectivity -n id-appgw-kv --query id -o tsv)
UAMI_PRINCIPAL=$(az identity show -g rg-connectivity -n id-appgw-kv --query principalId -o tsv)
UAMI_CLIENT=$(az identity show -g rg-connectivity -n id-appgw-kv --query clientId -o tsv)
echo "resourceId=$UAMI_ID"      # attach THIS to the gateway
echo "principalId=$UAMI_PRINCIPAL"  # grant the role to THIS

One subtle propagation detail that wastes time: a freshly-created UAMI’s service principal can take a short while to replicate across Entra. If you create the identity and immediately create a role assignment by principalId, you may get a transient PrincipalNotFound. The fix is to retry for a minute, or pass --assignee-principal-type ServicePrincipal (which tells the role-assignment API not to try to resolve the object as a user/group first) — covered in the failure-modes section.

The authorization model: cross-subscription RBAC (and access policies)

This is the step the portal cannot do and the conceptual heart of “cross-subscription.” You grant the gateway’s UAMI permission to read secrets on the central vault, scoped to the vault’s resource ID (which lives in the Identity subscription) even though the principal lives in Connectivity. The assignment is created in the vault’s subscription because the scope is the vault.

Two permission models exist on Key Vault; you grant in whichever the vault uses. Modern vaults use Azure RBAC; older ones still use vault access policies. (The trade-offs between the two are covered in Key Vault RBAC vs access policies; here you simply match the vault’s mode.)

Permission model How you grant secret-GET to the UAMI Scope you target Propagation Notes
Azure RBAC (recommended) az role assignment create --role "Key Vault Secrets User" The vault’s resource ID Seconds to a few minutes Least privilege; auditable in Activity log; the modern default
Vault access policy (legacy) az keyvault set-policy --secret-permissions get The vault (object-id based) Near-immediate Coarser; max ~1024 policies/vault; being deprecated in favour of RBAC
Both enabled (enableRbacAuthorization=true) RBAC wins; access policies ignored The vault Don’t mix mental models — check enableRbacAuthorization first

The least-privilege role is Key Vault Secrets User — GET on secrets, nothing else. Resist granting Key Vault Administrator or Contributor “to make it work”; the gateway needs one verb on one object type, and over-granting on your central secrets store is the opposite of what centralization is for. The RBAC grant, scoped to the central vault, created in the Identity subscription:

# Resolve the central vault's resource ID (it lives in the Identity subscription)
KV_ID=$(az keyvault show --subscription "$IDENTITY_SUB" -n kv-central-certs --query id -o tsv)

# Grant the UAMI's principal GET-on-secrets, scoped to that vault.
# --assignee-principal-type ServicePrincipal avoids a transient PrincipalNotFound on a new UAMI.
az role assignment create \
  --assignee-object-id "$UAMI_PRINCIPAL" \
  --assignee-principal-type ServicePrincipal \
  --role "Key Vault Secrets User" \
  --scope "$KV_ID"

If the vault is still on access policies (enableRbacAuthorization is false or unset), grant there instead — note you target the vault by name in the Identity subscription and grant only get:

# Legacy access-policy vault: grant ONLY secret get to the UAMI's principal
az keyvault set-policy --subscription "$IDENTITY_SUB" -n kv-central-certs \
  --object-id "$UAMI_PRINCIPAL" --secret-permissions get

Confirm which model the vault uses before you choose — granting an RBAC role to a policy-mode vault (or a policy to an RBAC-mode vault) silently does nothing useful:

az keyvault show --subscription "$IDENTITY_SUB" -n kv-central-certs \
  --query "properties.enableRbacAuthorization" -o tsv   # true → RBAC, false/null → access policies

The scope decision: vault, resource group, or single secret?

RBAC scope is a spectrum, and for this pattern the right default is the vault. Scoping wider (RG/subscription) grants the gateway read on every secret in that scope — over-broad on a central secrets store; scoping to a single secret via ABAC conditions is possible but fiddly and breaks if you rename or re-create the cert.

Scope What the UAMI can read When to use Risk
Subscription / RG Every secret in every vault in scope Almost never for this Massive over-grant on central secrets
The vault (recommended) Every secret in the central vault The normal choice UAMI can read other secrets in that vault
Single secret (ABAC condition) Only the wildcard secret Highly regulated estates Conditions are fiddly; breaks on rename
Per-cert dedicated vault Only that vault’s contents When you want hard isolation More vaults to govern

If your central vault holds only certificates, vault-scope is clean. If it co-mingles certs with database passwords and API keys, consider a dedicated certs vault so vault-scope and least-privilege coincide — cleaner than ABAC gymnastics.

Confirming the grant actually landed

Before you touch the gateway, prove the assignment exists at the vault scope and names your UAMI — a thirty-second check that prevents the most common “cert won’t load” incident:

# List role assignments at the vault scope; filter to our UAMI principal
az role assignment list --scope "$KV_ID" --assignee "$UAMI_PRINCIPAL" \
  --query "[].{role:roleDefinitionName, scope:scope, principal:principalId}" -o table
# Expect exactly: Key Vault Secrets User, scope = the central vault, principal = $UAMI_PRINCIPAL

If that returns empty, the gateway fails with a Forbidden/identity error no matter how perfect the rest is. (For deeper RBAC diagnosis — effective permissions, inherited and deny assignments — see troubleshooting AuthorizationFailed and managed-identity token-acquisition 403s.)

Binding the certificate to the listener (CLI)

With the identity created and granted, the gateway-side wiring is three moves: attach the UAMI to the gateway, create an SSL-cert object on the gateway that points at the unversioned secret ID, and point the HTTPS listener at that SSL-cert object.

The full move sequence, with the subscription each command targets — note that only the grant touches the Identity subscription, everything else is in Connectivity:

# Move Command (verb) Subscription Why it’s here
1 Create the UAMI az identity create Connectivity The principal the gateway authenticates as
2 Grant it on the vault az role assignment create --scope $KV_ID Identity (scope is the vault) Cross-subscription authorization
3 Open the network path PE + DNS, or --bypass AzureServices Identity (vault) / hub So the fetch can connect
4 Attach the UAMI to the gateway az network application-gateway identity assign Connectivity Gives the gateway a principal
5 Derive the unversioned secret id ${RAW%/*} Identity (read) The load-bearing correctness step
6 Create the SSL cert on the gateway az ... ssl-cert create --key-vault-secret-id Connectivity Binds the gateway to the secret
7 Point the listener at it az ... http-listener update --ssl-cert Connectivity Serves the cert on 443

Attach the identity. The gateway must carry the UAMI so it has a principal to authenticate as:

az account set --subscription "$CONNECTIVITY_SUB"
az network application-gateway identity assign \
  -g rg-connectivity --gateway-name agw-hub --identity "$UAMI_ID"

# Confirm it's attached
az network application-gateway identity show -g rg-connectivity --gateway-name agw-hub -o jsonc

Derive the unversioned secret ID. This is the load-bearing line. az keyvault secret show --query id returns the versioned URL; strip the trailing version segment so the binding tracks the current version forever:

RAW=$(az keyvault secret show --subscription "$IDENTITY_SUB" \
  --vault-name kv-central-certs -n wildcard-contoso --query id -o tsv)
echo "$RAW"                 # .../secrets/wildcard-contoso/8a1b2c3d...   <- VERSIONED, do NOT use
SECRET_ID="${RAW%/*}"       # .../secrets/wildcard-contoso               <- UNVERSIONED, USE THIS
echo "$SECRET_ID"

Create the SSL cert on the gateway and bind the listener:

# Create the gateway-side SSL cert object pointing at the unversioned secret id
az network application-gateway ssl-cert create \
  -g rg-connectivity --gateway-name agw-hub \
  -n wildcard-contoso --key-vault-secret-id "$SECRET_ID"

# Point the HTTPS listener at it
az network application-gateway http-listener update \
  -g rg-connectivity --gateway-name agw-hub -n https-listener \
  --ssl-cert wildcard-contoso

That is the whole fix. The gateway now serves the central cert and re-pulls automatically when it is renewed in the Identity subscription. If the listener does not yet exist (fresh gateway), create the frontend port (443), the SSL cert, and the listener in sequence:

az network application-gateway frontend-port create \
  -g rg-connectivity --gateway-name agw-hub -n port-443 --port 443

az network application-gateway http-listener create \
  -g rg-connectivity --gateway-name agw-hub -n https-listener \
  --frontend-port port-443 --frontend-ip appGwPublicFrontendIp \
  --ssl-cert wildcard-contoso --host-name app.contoso.com

Verify the binding is correct (and unversioned)

Three checks after binding, in order — under a minute, and they catch the three things that actually go wrong (versioned binding, cert not provisioned, served cert stale):

# 1) Inspect the SSL cert on the gateway — confirm the keyVaultSecretId is UNVERSIONED
az network application-gateway ssl-cert show -g rg-connectivity \
  --gateway-name agw-hub -n wildcard-contoso \
  --query "{name:name, kvSecretId:keyVaultSecretId, provisioningState:provisioningState}" -o jsonc
# kvSecretId MUST end in /secrets/wildcard-contoso  (no trailing GUID)

# 2) Confirm the gateway's overall operational/provisioning state is healthy
az network application-gateway show -g rg-connectivity -n agw-hub \
  --query "{op:operationalState, prov:provisioningState}" -o table

# 3) From a client, confirm the served cert's identity and validity window
curl -vI https://app.contoso.com 2>&1 | grep -Ei "subject:|issuer:|expire|start date|SSL certificate verify"

The most important assertion is the first one: kvSecretId must end in /secrets/<name> with no trailing GUID. If it ends in a GUID, you bound a versioned ID and auto-rotation is silently broken — re-run the bind against ${RAW%/*}.

A pre-flight checklist as a table — run top to bottom before you call the binding done:

Check Command / location Pass looks like
UAMI attached to the gateway az network application-gateway identity show Shows id-appgw-kv
Role granted at vault scope az role assignment list --scope $KV_ID --assignee $UAMI_PRINCIPAL One Key Vault Secrets User row
Binding is unversioned ssl-cert show --query keyVaultSecretId Ends in /secrets/<name> (no GUID)
SSL cert provisioned ssl-cert show --query provisioningState Succeeded
Vault firewall path open KV networking + nslookup from VNet PE private IP, or trusted-services on
Gateway running show --query operationalState Running
Served cert valid curl -vI https://<host> notAfter in the future, right subject

The status fields you may see and what they mean:

Field / value Meaning Action
provisioningState: Succeeded (ssl-cert) The cert object provisioned on the gateway Good
provisioningState: Failed (ssl-cert) The gateway couldn’t load the secret Check RBAC, network path, secret enabled
Cert state Unknown (portal/health) Authenticated but couldn’t fetch (network/firewall) Open the network path; check KV firewall
operationalState: Running Gateway is serving Good
operationalState: Stopped Gateway stopped (cost-saving or error) Start it; check why it stopped
Served cert’s notAfter in the past Stale/frozen version being served Likely versioned binding or rotation lag

Auto-rotation: the poll, the version, and the timing

Centralized certs are only worth it if rotation flows to the gateway without a human touching it. It does — under one condition and with one timing caveat.

The condition: the listener’s SSL cert is bound by the unversioned secret ID. The gateway polls Key Vault (documented as roughly every 4 hours) and, for an unversioned binding, re-resolves to the latest version each poll. When your rotation process creates a new version — KV’s integrated-CA auto-renewal, an ACME automation, or a manual import — the next poll picks it up and the gateway serves the new cert. No redeploy, no listener edit, no downtime.

The timing caveat: because the poll is periodic, there is a lag of up to the poll interval between “new version exists” and “gateway serves it.” Usually harmless because you renew well before expiry, but an outage if you cut it too fine. The rotation timeline you must respect:

Phase What happens Who drives it Timing you control
Renewal trigger Vault/automation creates a new secret version KV auto-renew / ACME / manual Set renew at e.g. 30 days before expiry
New version notBefore The new cert becomes valid from this instant The issuing CA / your policy Ensure notBefore ≤ now at publish
Gateway poll AppGW re-fetches the unversioned secret (~every 4h) Platform Not tunable; budget for it
Gateway swaps live cert New cert served on the listener Platform Happens automatically post-poll
Old version notAfter Old cert expires The old cert Must be AFTER the swap, with margin

The cardinal rule that prevents the classic outage: renew with generous margin (e.g. 30+ days before expiry), never on the last day. If the new version’s notBefore is in the future, or the old version expires within one poll interval of publishing the new one, you can have a window where the gateway is still polling-toward the new cert while the old one has died — a self-inflicted expiry. Give the poll room.

Built-in auto-renewal vs external automation

Two common rotation engines, both compatible with the unversioned binding:

Rotation engine How a new version appears Pros Watch-outs
KV integrated CA (DigiCert/GlobalSign account in KV) KV auto-renews per the cert policy (e.g. at 80% lifetime / N days before expiry) Hands-off; lifecycle in one place Requires a CA account configured in KV; policy must set a sane renewal trigger
ACME / Let’s Encrypt automation Your function/runner imports a renewed PFX as a new version Free certs; full control You own the renewal job’s reliability and the notBefore
Manual import An operator imports a new version before expiry Simple Easy to forget; alert on expiry
Event Grid → automation KV emits NearExpiry/Expired; a handler renews Proactive, auditable One more moving part to keep healthy

Set the certificate policy’s renewal trigger explicitly on the integrated-CA path, so a new version is guaranteed to exist before the poll/expiry math gets tight:

# Example: tell KV to auto-renew this cert 30 days before expiry (integrated CA)
az keyvault certificate get-default-policy > policy.json
# edit policy.json: lifetime_actions -> action AutoRenew, trigger days_before_expiry: 30
az keyvault certificate create --subscription "$IDENTITY_SUB" \
  --vault-name kv-central-certs -n wildcard-contoso -p @policy.json

Regardless of engine, wire an alert on cert expiry as a backstop. Key Vault emits Event Grid events (CertificateNearExpiry, CertificateExpired) you route to a webhook/Logic App/queue (see Event Grid custom topics and subscriptions), or you alert on the cert-expiry metric in Azure Monitor (see Azure Monitor and Application Insights). A central cert that expires unnoticed takes down every gateway depending on it — the blast radius is the whole point of centralizing, and why the alert is non-negotiable.

Forcing a re-pull (when you cannot wait for the poll)

Sometimes — testing rotation, recovering from an incident — you need the new version now rather than waiting up to four hours. There is no “refresh cert” button; you nudge the gateway by re-asserting the binding (which triggers a re-fetch):

# Re-assert the (still unversioned) secret id to force an immediate re-fetch
az network application-gateway ssl-cert update \
  -g rg-connectivity --gateway-name agw-hub \
  -n wildcard-contoso --key-vault-secret-id "$SECRET_ID"

This is a poll-bypass, not a fix for a versioned binding — if the binding is versioned, re-asserting the same versioned URL changes nothing. Fix the binding to unversioned first, then (optionally) force the re-pull. Your options for getting a new version onto the gateway:

Method How it works Latency When to use
Wait for the poll Gateway re-fetches the unversioned id (~every 4h) Up to ~4h Normal operation; renew with margin
ssl-cert update (re-assert) Re-applies the binding → triggers a re-fetch Minutes Testing rotation; incident recovery
Any gateway config change A config apply re-evaluates the binding Minutes Side effect of an unrelated change
Fix a versioned binding Re-bind unversioned, then optionally force Minutes The actual fix when frozen

The same pattern across other consumer services

The identity-plus-secret-GET binding is not special to Application Gateway — it is how every Azure service that terminates TLS pulls a central cert. Build it here and you’ve built it everywhere; only the consumer resource and the field name change:

Consumer service Identity it uses Permission on the vault Cert reference Auto-rotation
Application Gateway v2 User-assigned MI (required) Key Vault Secrets User sslCertificate.keyVaultSecretId (unversioned) Poll ~4h
App Service / Functions System- or user-assigned MI Key Vault Secrets User Imported KV certificate binding Periodic refresh
Front Door Standard/Premium Front Door’s managed identity Key Vault Secrets User (or certs) Customer certificate from KV Managed rotation
API Management System- or user-assigned MI Key Vault Secrets User KV-referenced custom-domain cert Periodic refresh (configurable)

The cross-subscription mechanics — scope the role at the vault’s resource ID, bind the unversioned secret, open the network path — are identical across all four. Standardize the certs vault, the UAMI grant, and the unversioned-binding guardrail once, and every consumer inherits it.

The network path: firewall, Private Endpoint, and trusted services

RBAC lets the gateway authenticate to Key Vault; the network still has to let it reach the vault’s data-plane endpoint. On a hardened central vault the firewall is “selected networks” (default-deny), and a gateway that authenticates but cannot connect shows the cert in state Unknown. You must deliberately open one path — and the choice between the two supported approaches has real consequences:

Approach Mechanism Network exposure When to use Caveats
Private Endpoint to KV (recommended) A private IP for the vault in a subnet the gateway can route to, + private DNS Vault data plane stays off the public internet Default for production landing zones Needs PE + privatelink.vaultcore.azure.net DNS resolving for the gateway
“Allow trusted Microsoft services” + public endpoint KV firewall exception that trusts certain Azure first-party services Vault public endpoint reachable (restricted) Simple estates without full private networking App Gateway must be a trusted service; broader than a PE; some orgs forbid public KV entirely
IP allow-list of the gateway’s outbound IP KV firewall rule for the gateway’s public egress IP Vault public endpoint, narrowly Edge cases; not preferred Outbound IP can change; brittle; still public
Service endpoint for KV on the gateway subnet Microsoft.KeyVault service endpoint on the subnet + KV network rule Optimized public route, not private IP Legacy; PE is the modern choice Not a private IP; see PE-vs-SE guidance

The decision in one line: use a Private Endpoint for production, reserve the trusted-services exception for simpler/transitional estates. Background: Private Endpoint vs Service Endpoint, the decision guide, and the DNS-at-scale story (where Private Endpoints most often go wrong).

Private Endpoint to Key Vault, the right way

A Key Vault Private Endpoint puts a private IP for the vault into a subnet, and a private DNS zone (privatelink.vaultcore.azure.net) makes the vault’s public hostname resolve to that private IP for anything that uses the zone. For the gateway to use it, two things must be true: its VNet must route to the PE’s subnet (in hub-and-spoke, the PE usually sits in the hub or a peered shared-services VNet), and its VNet must resolve the privatelink zone (linked to the VNet, or fronted by a central DNS/Private Resolver).

# Create a Private Endpoint for the central vault, in a subnet routable from the gateway.
# (Run in the subscription that owns the target VNet/subnet — often Connectivity/hub.)
az network private-endpoint create \
  -g rg-connectivity -n pe-kv-central -l centralindia \
  --vnet-name vnet-hub --subnet snet-privateendpoints \
  --private-connection-resource-id "$KV_ID" \
  --group-id vault \
  --connection-name kv-central-conn

# Private DNS zone for Key Vault privatelink, linked to the gateway's VNet, with an A record
az network private-dns zone create -g rg-connectivity -n privatelink.vaultcore.azure.net
az network private-dns link vnet create -g rg-connectivity -n link-hub \
  -z privatelink.vaultcore.azure.net --virtual-network vnet-hub --registration-enabled false
az network private-endpoint dns-zone-group create \
  -g rg-connectivity --endpoint-name pe-kv-central -n zg-kv \
  --private-dns-zone privatelink.vaultcore.azure.net --zone-name vault

Then set the vault firewall to deny public access (the PE is the way in). Critically, keep bypass: AzureServices on so the Application Gateway integration is treated as a trusted Azure-service path even with default-deny — the most reliable combination for AppGW + KV:

az keyvault update --subscription "$IDENTITY_SUB" -n kv-central-certs \
  --default-action Deny --bypass AzureServices

The single most common Private-Endpoint failure here is DNS: if the gateway’s VNet does not resolve privatelink.vaultcore.azure.net, the vault hostname resolves to its public IP, the firewall (default-deny) refuses it, and the cert goes Unknown — looking exactly like an RBAC problem when it is a name-resolution problem. The DNS resolution outcomes and what each means:

DNS setup for the gateway’s VNet nslookup <vault>.vault.azure.net returns Fetch via PE works? Verdict
Privatelink zone linked to the VNet, PE A-record present Private IP (10.x) Yes Correct
Privatelink zone exists but not linked to this VNet Public Microsoft IP No (firewall denies) The classic trap — link the zone
Central DNS / Private Resolver fronts the VNet Private IP (10.x) Yes Correct (hub-and-spoke at scale)
No private DNS at all Public IP No Add the zone or use trusted-services
Custom DNS server not forwarding privatelink Public IP / NXDOMAIN No Fix the conditional forwarder

Confirm resolution from inside the gateway’s network boundary (e.g. a test VM/NIC in the same VNet):

# From a VM in the gateway's VNet (or via a connection-monitor test):
nslookup kv-central-certs.vault.azure.net
# Must resolve to the PRIVATE IP (10.x), not a public Microsoft IP.

The “Allow trusted Microsoft services” path

If you are not yet doing full private networking, the simpler path is to keep the vault on its public endpoint with default-deny, and rely on the trusted Microsoft services exception so the gateway’s first-party integration is allowed through:

az keyvault update --subscription "$IDENTITY_SUB" -n kv-central-certs \
  --default-action Deny --bypass AzureServices

This is broader than a Private Endpoint (the vault’s public endpoint is still reachable, just restricted), and some regulated estates forbid any public Key Vault endpoint at all — there the Private Endpoint path is mandatory. Use trusted-services for transitional or lower-sensitivity estates; move to Private Endpoint as the steady state. The firewall settings that interact with this integration:

KV firewall setting Values Effect on AppGW fetch Recommended for this pattern
defaultAction Allow / Deny Deny = default-deny public; only PE / trusted-services / allow-listed IPs get in Deny (with a path opened)
bypass AzureServices / None AzureServices lets trusted first-party services through default-deny AzureServices
ipRules CIDRs Allow specific public IPs (e.g. gateway egress) Avoid; brittle
virtualNetworkRules Subnet IDs (service endpoint) Allow a subnet via service endpoint Legacy; prefer PE
Private endpoint connections Approved PEs The private path in Yes for production

A note on the gateway subnet: Application Gateway v2 lives in its own dedicated subnet, which needs outbound reachability appropriate to your path (route to the PE subnet for private; outbound internet/Service-Tag for trusted-services public). If you run forced-tunneling or restrictive NSGs/UDRs there, make sure the route to Key Vault is not blocked — a UDR sending 0.0.0.0/0 to a firewall that drops Key Vault traffic breaks the fetch just as surely as a missing role.

The same thing in Terraform (the version you actually commit)

The CLI is for understanding and incidents; the version you live with is in code, reviewed in a PR, with the binding’s unversioned-ness enforceable in CI. The Terraform crux is the provider alias: the identity and gateway are created in Connectivity, but the role assignment must be created in the Identity subscription. Configure two azurerm providers and point the role assignment at the aliased one.

# Two providers: default = Connectivity, alias "identity" = the vault's subscription
provider "azurerm" {
  features {}
  subscription_id = var.connectivity_sub
}

provider "azurerm" {
  features {}
  alias           = "identity"
  subscription_id = var.identity_sub
}

# The central vault lives in the Identity subscription — read it via the aliased provider
data "azurerm_key_vault" "central" {
  provider            = azurerm.identity
  name                = "kv-central-certs"
  resource_group_name = var.identity_kv_rg
}

# The certificate's UNVERSIONED secret id (the .secret_id attribute is unversioned)
data "azurerm_key_vault_certificate" "wildcard" {
  provider     = azurerm.identity
  name         = "wildcard-contoso"
  key_vault_id = data.azurerm_key_vault.central.id
}

# User-assigned identity in Connectivity
resource "azurerm_user_assigned_identity" "appgw" {
  name                = "id-appgw-kv"
  resource_group_name = var.connectivity_rg
  location            = var.location
}

# Cross-subscription role assignment: created in the Identity subscription via the alias,
# scoped to the central vault, granting the UAMI GET-on-secrets.
resource "azurerm_role_assignment" "kv_get" {
  provider             = azurerm.identity
  scope                = data.azurerm_key_vault.central.id
  role_definition_name = "Key Vault Secrets User"
  principal_id         = azurerm_user_assigned_identity.appgw.principal_id
}

resource "azurerm_application_gateway" "hub" {
  name                = "agw-hub"
  resource_group_name = var.connectivity_rg
  location            = var.location

  sku {
    name = "WAF_v2"
    tier = "WAF_v2"
  }
  autoscale_configuration {
    min_capacity = 2
    max_capacity = 10
  }

  identity {
    type         = "UserAssigned"
    identity_ids = [azurerm_user_assigned_identity.appgw.id]
  }

  ssl_certificate {
    name = "wildcard-contoso"
    # .secret_id on the certificate data source is the UNVERSIONED secret id — exactly what we want
    key_vault_secret_id = data.azurerm_key_vault_certificate.wildcard.secret_id
  }

  http_listener {
    name                           = "https-listener"
    frontend_ip_configuration_name = "appGwPublicFrontendIp"
    frontend_port_name             = "port-443"
    protocol                       = "Https"
    ssl_certificate_name           = "wildcard-contoso"
    host_name                      = "app.contoso.com"
  }

  # ... gateway_ip_configuration, frontend_ip_configuration, frontend_port (443),
  #     backend_address_pool, backend_http_settings, request_routing_rule ...

  # Force the grant to exist before the gateway tries to read the cert
  depends_on = [azurerm_role_assignment.kv_get]
}

Three Terraform-specific points that save real grief:

Concern The gotcha The fix in code
Unversioned binding Using .versionless_secret_id/raw secret id wrong yields a pinned version Use data.azurerm_key_vault_certificate.wildcard.secret_id (already unversioned)
Ordering Gateway provisions before the role exists → cert load fails on first apply depends_on = [azurerm_role_assignment.kv_get]
Cross-sub auth The principal running terraform apply needs rights in both subs RBAC to write the role assignment in Identity + create resources in Connectivity
Drift on rotation After a renewal, the data source’s secret_id is unchanged (it’s unversioned) No drift — that’s the point; the live cert changes, the config doesn’t

Enforce the unversioned rule as a policy-as-code gate — an OPA/conftest rule that fails any plan whose key_vault_secret_id matches a trailing version GUID. One ambiguous portal URL silently defeating auto-rotation is exactly what a CI guardrail should catch before production:

# conftest/OPA: deny a versioned key_vault_secret_id on an Application Gateway ssl_certificate
package main

deny[msg] {
  rc := input.resource_changes[_]
  rc.type == "azurerm_application_gateway"
  cert := rc.change.after.ssl_certificate[_]
  re_match(`/secrets/[^/]+/[0-9a-f]{32}$`, cert.key_vault_secret_id)
  msg := sprintf("ssl_certificate '%s' uses a VERSIONED secret id — auto-rotation will break", [cert.name])
}

(If you prefer reusable modules over hand-rolled HCL, the building blocks exist as the Application Gateway module, the Key Vault module, the managed-identity module, the RBAC role-assignment module, and the Private Endpoint module — wire them with the same provider-alias pattern.)

The same thing in Bicep / ARM

Bicep is the native-Azure path. Cross-subscription is handled by deploying the role assignment at the vault’s scope in the Identity subscription via a module targeting that subscription/RG. The gateway and identity deploy in Connectivity; the role assignment deploys against the vault.

// appgw-kv.bicep — deployed into the Connectivity resource group
param location string = resourceGroup().location
param identitySubId string
param identityKvRg string
param kvName string = 'kv-central-certs'
param certName string = 'wildcard-contoso'

// User-assigned identity in Connectivity
resource uami 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = {
  name: 'id-appgw-kv'
  location: location
}

// Reference the central vault (in the Identity subscription) to read its secret URI
resource kv 'Microsoft.KeyVault/vaults@2023-07-01' existing = {
  name: kvName
  scope: resourceGroup(identitySubId, identityKvRg)
}

// Grant the UAMI 'Key Vault Secrets User' on the central vault, via a module
// scoped to the Identity subscription/RG (cross-subscription role assignment).
module kvGrant 'kv-grant.bicep' = {
  name: 'grant-appgw-kv'
  scope: resourceGroup(identitySubId, identityKvRg)
  params: {
    kvName: kvName
    principalId: uami.properties.principalId
  }
}

resource agw 'Microsoft.Network/applicationGateways@2023-11-01' = {
  name: 'agw-hub'
  location: location
  identity: {
    type: 'UserAssigned'
    userAssignedIdentities: {
      '${uami.id}': {}
    }
  }
  properties: {
    sku: { name: 'WAF_v2', tier: 'WAF_v2', capacity: 2 }
    sslCertificates: [
      {
        name: certName
        properties: {
          // Unversioned secret id → auto-rotation. Construct it WITHOUT a version.
          keyVaultSecretId: '${kv.properties.vaultUri}secrets/${certName}'
        }
      }
    ]
    // ... gatewayIPConfigurations, frontendIPConfigurations, frontendPorts (443),
    //     httpListeners (referencing the sslCertificate), backendAddressPools, rules ...
  }
  dependsOn: [ kvGrant ]
}
// kv-grant.bicep — deployed at the Identity subscription's KV resource group
param kvName string
param principalId string

resource kv 'Microsoft.KeyVault/vaults@2023-07-01' existing = {
  name: kvName
}

resource secretsUser 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(kv.id, principalId, 'kv-secrets-user')
  scope: kv
  properties: {
    // Key Vault Secrets User
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions',
      '4633458b-17de-408a-b874-0445c86b69e6')
    principalId: principalId
    principalType: 'ServicePrincipal'
  }
}

The Bicep crux mirrors Terraform: construct keyVaultSecretId as ${vaultUri}secrets/${certName} with no version, and deploy the role assignment at the vault’s scope in the other subscription via a module with scope: resourceGroup(identitySubId, identityKvRg), with dependsOn: [kvGrant]. The IaC paths compared:

Aspect az CLI Bicep / ARM Terraform
Cross-sub role assignment --scope = vault id (any sub) Module scope: resourceGroup(otherSub, rg) provider = azurerm.identity alias
Unversioned secret id ${RAW%/*} by hand ${vaultUri}secrets/<name> constructed data...certificate.secret_id (already unversioned)
Ordering guarantee You sequence the commands dependsOn: [kvGrant] depends_on = [role_assignment]
Drift after rotation N/A (imperative) None (unversioned literal) None (unversioned data attr)
CI guardrail Lint the script ARM-TTK / PSRule conftest/OPA on the plan
Best for Learning, incidents Azure-native shops Multi-cloud / module estates

Architecture at a glance

Walk the request and the trust the way the system actually executes it. A client opens https://app.contoso.com; DNS points the hostname at the Application Gateway’s public frontend IP in the Connectivity subscription. The gateway’s HTTPS listener on port 443 must present the wildcard cert to complete the handshake — and that cert lives not with the gateway but in kv-central-certs, a Key Vault in the Identity subscription, as a secret holding the full PFX. To fetch it, the gateway authenticates as its user-assigned identity id-appgw-kv, which holds one cross-subscription role assignment — Key Vault Secrets User, scoped to the central vault — letting it GET /secrets/wildcard-contoso. The vault firewall is default-deny, so the fetch travels the Private Endpoint path (a private IP in the hub, resolved via the privatelink.vaultcore.azure.net zone) rather than the internet. The gateway loads the cert, terminates TLS, applies its WAF, and routes the decrypted request over the hub to the workload in a spoke subscription. Every few hours it re-polls the unversioned secret URL, so a renewal in the central vault is served transparently.

The diagram below shows that whole topology as zoned panels — the three subscriptions, the identity arrow from gateway to UAMI, the RBAC arrow from UAMI to the vault, the Private-Endpoint network path, and the auto-rotation poll — so you can trace both the trust path (who is allowed) and the network path (how the bytes travel), the two independent gates this pattern hinges on.

Cross-subscription topology: an Application Gateway with a user-assigned managed identity in the Connectivity subscription pulling a wildcard TLS certificate from a central Key Vault in the Identity subscription via a Key Vault Secrets User role assignment and a Private Endpoint, terminating TLS on its HTTPS listener and routing to workloads in spoke subscriptions, with an auto-rotation poll loop

Read it left to right as the build order, too: create the identity (Connectivity), grant it on the vault (Identity), open the network path (Private Endpoint + DNS), then bind the listener by the unversioned secret ID (Connectivity). Four moves across three subscriptions — and the portal can only do the ones inside the gateway’s own subscription, which is precisely why this is a code pattern.

Real-world scenario

Contoso Retail ran a textbook Enterprise-Scale landing zone: a Connectivity subscription with a hub VNet and a WAF_v2 Application Gateway fronting every public workload, an Identity subscription holding kv-central-certs with the *.contoso.com wildcard from their corporate CA, and a dozen spoke subscriptions for the apps. The platform team wired the listener to the central cert at go-live exactly as this article prescribes — UAMI, Key Vault Secrets User, Private Endpoint — and it served traffic cleanly for six weeks.

Then monitoring lit up: browsers were getting NET::ERR_CERT_DATE_INVALID on app.contoso.com — the gateway was serving an expired wildcard. The on-call engineer’s first instinct was a vault problem, but az keyvault certificate show proved the cert had been renewed on schedule: a fresh, valid version sat in the vault, notBefore weeks in the past, notAfter a year out. The renewal worked perfectly; the gateway just never picked it up.

The root cause was a versioned binding. During a mid-project change, an engineer re-bound the listener’s SSL cert by copying the full secret URL out of the portal — version GUID and all. App Gateway only auto-rotates against an unversioned reference; a pinned version is frozen forever. The cert renewed under a new version; the gateway, still polling the old pinned version, kept fetching the now-expired one; and nothing in the portal surfaced the mismatch until the pinned version’s notAfter passed and users hit the browser error. The very failure mode centralized certs are supposed to prevent had been silently re-introduced by one ambiguous copy-paste.

The fix took two minutes: re-derive the unversioned secret ID and re-bind.

RAW=$(az keyvault secret show --subscription "$IDENTITY_SUB" \
  --vault-name kv-central-certs -n wildcard-contoso --query id -o tsv)
SECRET_ID="${RAW%/*}"   # strip the version

az network application-gateway ssl-cert update \
  -g rg-connectivity --gateway-name agw-hub \
  -n wildcard-contoso --key-vault-secret-id "$SECRET_ID"

# Confirm the binding is now unversioned and re-fetched
az network application-gateway ssl-cert show -g rg-connectivity \
  --gateway-name agw-hub -n wildcard-contoso \
  --query "keyVaultSecretId" -o tsv   # ends in /secrets/wildcard-contoso, no GUID

Within minutes the gateway re-pulled the current version and the errors cleared. The lasting fix was process, not a command: they moved the binding into Terraform (where the secret_id data attribute is unversioned by construction), added an OPA/conftest guardrail that fails any plan with a trailing-GUID key_vault_secret_id, and wired a Key Vault CertificateNearExpiry alert as a backstop. Contoso’s incident is the canonical lesson: the wiring is easy; the correctness of the binding — unversioned, plus an expiry alarm — is what keeps a central cert from taking down the entire estate at once.

Advantages and disadvantages

The two-column trade-off of binding a gateway to a central, cross-subscription vault rather than copying certs into per-subscription vaults:

Advantages Disadvantages
One vault to rotate, audit, and govern — a single source of truth A central vault is a blast-radius concentration: its outage/expiry hits every gateway
Auto-rotation flows to every gateway with no per-gateway action Auto-rotation only works if every binding is unversioned (a silent footgun)
Least-privilege: each gateway’s UAMI gets GET on secrets, nothing more More moving parts to wire (UAMI, cross-sub RBAC, PE, DNS) than a local copy
No secret sprawl — the private key lives in one place, never copied The portal can’t do it; you must use CLI/IaC, raising the skill floor
Clean separation of duties (Security owns the vault; Network owns the gateway) Cross-subscription RBAC requires rights in two subscriptions to provision
Generalizes to Front Door / APIM / App Service with the same identity pattern Network path adds a Private Endpoint + private DNS to get right (DNS is the usual trap)
IaC-friendly and CI-guardable (enforce unversioned in a policy check) Rotation lag (poll interval) means “instant” is hours; tight expiry windows bite

When the central pattern clearly wins: any estate with more than one or two gateways/front-doors, regulated environments needing single-point audit and least privilege, and platforms requiring separation of duties between security and networking teams. A local copy is defensible only for a single isolated app in one subscription with no shared cert — and even then, the moment a second consumer appears, you want the central pattern. The dangerous middle ground is “we centralized but bound it versioned,” which gives the blast radius of centralization with the staleness of a manual copy. Get the binding unversioned and the alarm wired, and the advantages dominate.

Hands-on lab

Build the cross-subscription binding end to end and prove auto-rotation, then deliberately break it the way Contoso did and fix it — so you’ve felt the failure, not just read about it. Everything is free-tier-friendly except the gateway (no free tier; a Standard_v2 for an hour is a few rupees — delete at the end). Run in Cloud Shell (Bash).

Note: this lab uses two subscriptions to be faithful to the pattern. With only one, set both variables to the same subscription ID — the mechanics (UAMI, Key Vault Secrets User, unversioned binding, rotation) are identical; only the “cross-subscription” framing collapses.

Step 1 — Variables. Use real subscription IDs (or the same one twice).

IDENTITY_SUB="<identity-subscription-id>"
CONNECTIVITY_SUB="<connectivity-subscription-id>"
LOC=centralindia
RG_ID=rg-kvcert-lab-identity
RG_CONN=rg-kvcert-lab-conn
KV=kv-lab-$RANDOM            # globally-unique vault name
CERT=wildcard-lab

Step 2 — Central vault + a self-signed cert (Identity subscription). Create the vault in RBAC mode and a self-signed cert to stand in for the wildcard.

az account set --subscription "$IDENTITY_SUB"
az group create -n $RG_ID -l $LOC -o table
az keyvault create -n $KV -g $RG_ID -l $LOC --enable-rbac-authorization true -o table

# Grant YOURSELF certs officer so you can create the cert (RBAC vault)
ME=$(az ad signed-in-user show --query id -o tsv)
KV_ID=$(az keyvault show -n $KV -g $RG_ID --query id -o tsv)
az role assignment create --assignee-object-id "$ME" --assignee-principal-type User \
  --role "Key Vault Certificates Officer" --scope "$KV_ID"
sleep 30   # let the role propagate

# Create a self-signed cert for app.lab.local (stand-in for the wildcard)
az keyvault certificate create --vault-name $KV -n $CERT \
  -p "$(az keyvault certificate get-default-policy)" -o table

Expected: a vault row, then a certificate whose status becomes completed.

Step 3 — UAMI + cross-subscription grant (Connectivity → vault in Identity).

az account set --subscription "$CONNECTIVITY_SUB"
az group create -n $RG_CONN -l $LOC -o table
az identity create -g $RG_CONN -n id-appgw-lab -l $LOC -o table

UAMI_ID=$(az identity show -g $RG_CONN -n id-appgw-lab --query id -o tsv)
UAMI_PRINCIPAL=$(az identity show -g $RG_CONN -n id-appgw-lab --query principalId -o tsv)

# Cross-subscription role assignment: scope is the vault in the IDENTITY sub
az role assignment create \
  --assignee-object-id "$UAMI_PRINCIPAL" --assignee-principal-type ServicePrincipal \
  --role "Key Vault Secrets User" --scope "$KV_ID"

# Prove the grant landed
az role assignment list --scope "$KV_ID" --assignee "$UAMI_PRINCIPAL" \
  --query "[].{role:roleDefinitionName, scope:scope}" -o table

Expected: one row, Key Vault Secrets User, scoped to the lab vault.

Step 4 — Network for the gateway (VNet + subnet + public IP).

az network vnet create -g $RG_CONN -n vnet-lab -l $LOC --address-prefixes 10.40.0.0/16 \
  --subnet-name snet-appgw --subnet-prefixes 10.40.1.0/24 -o table
az network public-ip create -g $RG_CONN -n pip-agw-lab -l $LOC --sku Standard --allocation-method Static -o table

Step 5 — Derive the UNVERSIONED secret id (the load-bearing step).

RAW=$(az keyvault secret show --subscription "$IDENTITY_SUB" --vault-name $KV -n $CERT --query id -o tsv)
echo "versioned:   $RAW"
SECRET_ID="${RAW%/*}"
echo "unversioned: $SECRET_ID"   # ends in /secrets/wildcard-lab — USE THIS

Step 6 — Create the gateway with the UAMI and bind the cert.

az network application-gateway create \
  -g $RG_CONN -n agw-lab -l $LOC \
  --sku Standard_v2 --capacity 1 \
  --vnet-name vnet-lab --subnet snet-appgw \
  --public-ip-address pip-agw-lab \
  --identity "$UAMI_ID" \
  --frontend-port 443 \
  --key-vault-secret-id "$SECRET_ID" \
  --ssl-certificate-name $CERT \
  --priority 100 -o table

This can take several minutes (gateways are slow to provision). Expected: a gateway with provisioningState: Succeeded.

Step 7 — Verify the binding is correct and unversioned.

az network application-gateway ssl-cert show -g $RG_CONN --gateway-name agw-lab -n $CERT \
  --query "{name:name, kvSecretId:keyVaultSecretId, prov:provisioningState}" -o jsonc
# kvSecretId MUST end in /secrets/wildcard-lab (no trailing GUID) and prov = Succeeded

Expected: kvSecretId ends in /secrets/wildcard-lab; prov: Succeeded. You have a cross-subscription cert binding working.

Step 8 — Prove auto-rotation tracks a new version. Create a new version of the cert and confirm the unversioned id still resolves to the latest:

az keyvault certificate create --subscription "$IDENTITY_SUB" --vault-name $KV -n $CERT \
  -p "$(az keyvault certificate get-default-policy)" -o table   # new version

# The unversioned secret id is unchanged; the gateway will re-fetch the latest on its next poll.
# Force an immediate re-pull rather than waiting for the ~4h poll:
az network application-gateway ssl-cert update -g $RG_CONN --gateway-name agw-lab \
  -n $CERT --key-vault-secret-id "$SECRET_ID"

The point: you changed nothing on the gateway’s config (the unversioned id is identical), yet the live cert now tracks the new version. That is auto-rotation.

Step 9 — Reproduce the Contoso bug, then fix it. Re-bind with the versioned id (the footgun), observe it’s pinned, then fix:

# Pin to a specific version (the mistake)
VER_ID=$(az keyvault secret show --subscription "$IDENTITY_SUB" --vault-name $KV -n $CERT --query id -o tsv)
az network application-gateway ssl-cert update -g $RG_CONN --gateway-name agw-lab -n $CERT --key-vault-secret-id "$VER_ID"
az network application-gateway ssl-cert show -g $RG_CONN --gateway-name agw-lab -n $CERT --query keyVaultSecretId -o tsv
# ^ now ends in a GUID — auto-rotation is BROKEN

# Fix: re-bind unversioned
az network application-gateway ssl-cert update -g $RG_CONN --gateway-name agw-lab -n $CERT --key-vault-secret-id "${VER_ID%/*}"
az network application-gateway ssl-cert show -g $RG_CONN --gateway-name agw-lab -n $CERT --query keyVaultSecretId -o tsv
# ^ ends in /secrets/wildcard-lab again — fixed

Validation checklist. You created a vault and cert in one subscription, an identity and gateway in another, granted the cross-subscription Key Vault Secrets User role scoped to the vault, bound the listener by the unversioned secret id, proved a new version flows through, and felt the versioned-binding bug and its fix. Mapped to what each step proves:

Step What you did What it proves Real-world analogue
3 Grant Key Vault Secrets User scoped to a vault in another sub Cross-subscription RBAC works; subscription lines don’t gate identity The “portal can’t, RBAC can” insight
5 Strip the version from the secret id The default --query id is versioned; you must strip it The single most important correctness step
7 Confirm kvSecretId ends in /secrets/<name> The binding is unversioned and provisioned The 30-second pre-prod check
8 New cert version, same unversioned id Auto-rotation tracks the latest version What central rotation buys you
9 Pin then unpin the version The versioned-binding bug is real and reversible The Contoso incident, in miniature

Cleanup (avoid lingering gateway charges). The gateway is the only non-trivial cost; delete both resource groups.

az group delete --subscription "$CONNECTIVITY_SUB" -n $RG_CONN --yes --no-wait
az group delete --subscription "$IDENTITY_SUB" -n $RG_ID --yes --no-wait

Cost note. A Standard_v2 gateway at capacity 1 for an hour is well under ₹50; the vault and cert are effectively free at this scale. Deleting both resource groups stops everything. (Remember the vault is soft-deleted for the retention window after deletion — if you re-run the lab with the same name you may hit a name collision; use a fresh $RANDOM name or recover/purge per the soft-delete section.)

Common mistakes & troubleshooting

This is the playbook — the part you bookmark. Cross-subscription Key Vault binding fails in a small number of distinct ways whose symptoms overlap (most show as “cert won’t load” or “cert state Unknown”), so the value is the confirm column that tells them apart. First the scannable table, then the expanded reasoning for the ones that bite hardest.

# Symptom Root cause Confirm (exact cmd / portal path) Fix
1 Gateway serves an expired cert though the vault holds a fresh one Versioned secret-id binding (frozen forever) az network application-gateway ssl-cert show --query keyVaultSecretId ends in a GUID Re-bind unversioned: --key-vault-secret-id "${RAW%/*}"
2 Cert won’t load; SSL cert provisioningState: Failed UAMI lacks Key Vault Secrets User on the vault az role assignment list --scope $KV_ID --assignee $UAMI_PRINCIPAL empty Grant the role scoped to the vault (cross-sub)
3 Cert state Unknown; auth seems fine KV firewall (default-deny) blocks the fetch KV networking = “selected networks”; no PE/trusted-services path Add PE + private DNS, or --bypass AzureServices
4 Cert Unknown; PE exists but still blocked Gateway VNet resolves the vault’s public IP (DNS) nslookup <vault>.vault.azure.net from gateway VNet returns a public IP Link privatelink.vaultcore.azure.net to the gateway VNet
5 Cert load fails with a Forbidden/permission error Granted certificates role, not secrets Role is Certificates User, binding is /secrets/... Grant Key Vault Secrets User (secrets GET)
6 ssl-cert create rejects the id / cert won’t load Bound a /certificates/ URL, not /secrets/ The bound id contains /certificates/ Use the secret id (az keyvault secret show), strip version
7 New UAMI grant fails with PrincipalNotFound Entra replication lag for the fresh principal Role assignment errors immediately after az identity create Add --assignee-principal-type ServicePrincipal; retry ~60s
8 Re-creating the vault/cert errors Conflict Soft-deleted name still reserved az keyvault list-deleted shows the name Recover (az keyvault recover) or purge if allowed
9 Cert load fails after a renewal that deleted the old cert New cert disabled, or wrong content type az keyvault certificate show --query attributes.enabled = false Enable the cert; ensure PKCS12/PEM content type
10 Rotation lag: new version exists but gateway serves old Poll interval (~4h) not yet elapsed New version notBefore recent; gateway not re-polled Wait for poll, or force re-pull via ssl-cert update
11 Self-inflicted expiry during rotation Old notAfter within one poll of new notBefore Renewal done on/near expiry day Renew 30+ days early; never on the last day
12 ssl-cert create fails: identity not usable System-assigned identity used (unsupported) Gateway identity.type is SystemAssigned only Attach a user-assigned identity
13 Portal listener picker can’t see the cross-sub vault Portal limitation (per-subscription picker) Vault is in another subscription Bind via CLI/Bicep/Terraform (not the portal)
14 Cert load fails behind a firewall/UDR on the AppGW subnet UDR/NSG drops the route to KV (public or PE) Effective routes on the AppGW subnet send KV traffic to a dropping NVA Allow the KV route; permit AzureKeyVault service tag / PE subnet
15 Vault has purge protection; can’t delete-recreate in CI Purge protection blocks early purge az keyvault show --query properties.enablePurgeProtection = true Don’t delete; version in place, or wait out retention

The expanded form, with the reasoning for the entries that bite hardest:

1. The gateway serves an expired certificate even though the vault was renewed on time. The listener’s SSL cert was bound by the versioned secret ID, freezing the gateway on one version; the renewal landed under a new version it never polls. Confirm: ssl-cert show --query keyVaultSecretId -o tsv ends in a 32-hex-char GUID. Fix: re-derive the unversioned id (${RAW%/*}), ssl-cert update, and add an OPA/conftest gate so a versioned id can’t be committed again.

2. The cert never loads; the SSL cert provisions Failed. The UAMI does not actually hold Key Vault Secrets User on the central vault — forgotten, scoped to the wrong vault, or granted to the gateway’s resource ID instead of the UAMI’s principalId. Confirm: az role assignment list --scope "$KV_ID" --assignee "$UAMI_PRINCIPAL" is empty; cross-check identity show. Fix: create the assignment with --assignee-principal-type ServicePrincipal, wait for propagation, force a re-pull.

3. The cert shows state Unknown — it authenticates but can’t fetch. The vault firewall is default-deny with no open network path; RBAC passed, the data-plane connection is refused. Confirm: KV → Networking shows “selected networks”, no approved PE, bypass: None; cert state is Unknown (not Failed). Fix: add a PE routable from the gateway and bypass: AzureServices; or set --default-action Deny --bypass AzureServices.

4. The Private Endpoint exists but the fetch is still blocked. DNS. The gateway’s VNet doesn’t resolve privatelink.vaultcore.azure.net, so the vault hostname resolves to its public IP and the default-deny firewall refuses it — identical-looking to an RBAC failure. Confirm: nslookup <vault>.vault.azure.net from the gateway’s VNet returns a public IP, not the PE’s 10.x. Fix: link the privatelink zone to the gateway’s VNet (or front it with the Private Resolver), and ensure the PE’s DNS zone group created the A record.

5 & 6. Permission/URL confusion: certificates vs secrets. Either you granted a certificates role (which doesn’t cover the /secrets/ GET the gateway does), or you bound the /certificates/ URL instead of the /secrets/ URL — the cert is read through its secret representation. Confirm the granted role is Key Vault Secrets User (not …Certificates User) and the bound id contains /secrets/. Fix: grant the secrets role; derive the binding from az keyvault secret show --query id and strip the version.

7. A brand-new identity’s role assignment fails with PrincipalNotFound. The UAMI’s service principal hasn’t replicated across Entra yet. It appears only immediately after az identity create; retrying succeeds. Fix: pass --assignee-object-id with --assignee-principal-type ServicePrincipal (skips the user/group lookup) and add a short retry/sleep in automation.

8 & 15. Soft-delete and purge-protection collisions on re-create. Soft-delete reserves the vault/cert name for the retention window; purge protection blocks purging it early. A delete-then-recreate (lab teardown, CI ephemeral env) collides with the soft-deleted shadow. Confirm with az keyvault list-deleted and az keyvault show --query properties.enablePurgeProtection. Fix: recover (az keyvault recover) or — if purge is permitted — purge; in CI, prefer versioning in place over delete-recreate. See the Key Vault soft-delete recovery deep dive.

10 & 11. Rotation lag and self-inflicted expiry. The gateway polls roughly every four hours, so a new version isn’t served instantly; if you renew on the expiry day, the old version can die before the next poll serves the new one. Confirm by comparing the new version’s notBefore and the old version’s notAfter against the poll window. Fix: renew with generous margin (30+ days); if you need it now, force a re-pull via ssl-cert update.

12. The gateway can’t use the identity for Key Vault. Only a system-assigned identity is attached; this integration requires a user-assigned one. Confirm with az network application-gateway identity show. Fix: create a UAMI, attach it, grant it on the vault, re-bind.

14. A firewall/UDR on the gateway subnet silently drops Key Vault traffic. A UDR sends 0.0.0.0/0 (or the KV path) to an NVA/Azure Firewall that doesn’t permit Key Vault, so even a correct PE/public path is dropped at the route layer. Confirm via effective routes on the gateway’s NIC and firewall drop logs. Fix: allow the AzureKeyVault service tag (public path) or the PE subnet (private path) through the NVA/UDR.

A compact error / status reference for the strings and states you’ll actually see, since several distinct causes share a surface:

Surface signal Where you see it Most likely cause(s) First confirm
SSL cert provisioningState: Failed ssl-cert show Missing role; bad/disabled secret; wrong URL role assignment list at vault scope
Cert state Unknown Portal cert status / health KV firewall block; DNS to public IP; UDR drop nslookup vault from gateway VNet
Forbidden / 403 on secret get Activity/diag logs Certificates role instead of secrets; firewall Role name; KV networking
PrincipalNotFound role assignment create Entra replication lag for new UAMI Retry with --assignee-principal-type ServicePrincipal
Conflict / ConflictError keyvault create / cert create Soft-deleted name reserved az keyvault list-deleted
NET::ERR_CERT_DATE_INVALID (browser) Client Versioned binding; rotation lag; expiry ssl-cert show --query keyVaultSecretId
Listener won’t pick cross-sub vault Portal Portal per-subscription picker limitation Use CLI/IaC
BadRequest on ssl-cert create CLI /certificates/ URL; system-assigned identity Inspect the bound id; identity type

Best practices

The platform/security guardrails worth standing up around this pattern:

Guardrail Mechanism Prevents Where it runs
Unversioned-binding gate OPA/conftest on the plan; PSRule on ARM Frozen-cert expiry (the Contoso bug) CI pipeline
Least-privilege enforcement Azure Policy: deny over-broad KV role assignments Over-granting on central secrets Subscription/MG
Vault default-deny Azure Policy: KV must have defaultAction: Deny Public-exposed central cert vault Subscription/MG
Expiry alarm Event Grid + Logic App / Monitor alert Unnoticed central-cert expiry Identity subscription
Purge-protection on Azure Policy: KV enablePurgeProtection: true Malicious/accidental cert destruction Subscription/MG
Binding as code PR review + drift detection Click-ops bindings that rot Git + CI

Security notes

The security controls and what each buys you here:

Control Setting / mechanism Secures against Also prevents
Secrets-only, vault-scoped role Key Vault Secrets User at vault scope Over-broad access to central secrets Accidental write/delete by the gateway identity
User-assigned identity UAMI + RBAC Credential sprawl / static secrets Re-granting on every gateway re-create
Default-deny + Private Endpoint KV defaultAction: Deny + PE Public exposure / exfiltration “Unknown cert” from open-but-misrouted vaults (when DNS is right)
bypass: AzureServices KV firewall Blocking the legit integration Over-opening IP rules to make it work
Soft-delete + purge protection KV data-protection Destruction of the cert Permanent loss; CI delete-recreate accidents
Diagnostic logging on secret-get KV diagnostics → Log Analytics Unaudited secret reads Blind spots during an incident
Expiry alarm Event Grid / Monitor Silent estate-wide outage Last-minute, error-driven discovery

Cost & sizing

The cost story is dominated by the Application Gateway, not the Key Vault binding — the binding itself is effectively free (RBAC + a secret read), and the vault costs a trivial amount per operation. The key facts:

A rough monthly picture for a single hub gateway fronting a few apps, with the central-cert pattern:

Cost driver What you pay for Rough INR / month Notes
Application Gateway WAF_v2 (small, continuous) Gateway hour + capacity units + WAF ~₹22,000–35,000 Dominant cost; exists regardless of cross-sub
Application Gateway Standard_v2 (no WAF) Gateway hour + capacity units ~₹18,000–28,000 Cheaper if you don’t need WAF
Key Vault operations (cert polls + app secrets) Per 10k operations (standard) ~₹0–200 The cert poll is negligible
Key Vault Private Endpoint Hourly + per-GB ~₹1,000–2,000 One PE, low volume
Private DNS zone Per zone + per-query ~₹50–150 Rounding error
UAMI + role assignment + binding ₹0 The cross-sub mechanics are free

Sizing guidance for the gateway sits in Application Gateway autoscaling and zone-redundant setup; the binding doesn’t change those numbers. The budgeting headline: centralizing the cert cross-subscription saves operational cost (one rotation, one audit) at no incremental Azure spend — the only money is the gateway and, optionally, the Private Endpoint, both of which you’d want anyway.

Interview & exam questions

1. Why can’t you select a Key Vault in another subscription from the Application Gateway listener UI, and does that mean cross-subscription certs are unsupported? The portal’s certificate picker only enumerates vaults in the gateway’s own subscription — a UI limitation, not a platform one. Cross-subscription binding works perfectly because RBAC scope (the vault’s resource ID) and identity (the UAMI’s principal) are both resolved tenant-wide; subscription lines don’t gate authorization. You wire it via CLI/ARM/Bicep/Terraform instead of the portal.

2. Which Key Vault object does Application Gateway read to get a certificate, and which permission does its identity need? It reads the secret representation (/secrets/<name>), which contains the full certificate including the private key (PFX/PEM), because terminating TLS needs the private key. The least-privilege role is Key Vault Secrets User (GET on secrets) — not a certificates role, which doesn’t cover the secret read.

3. Why must the managed identity be user-assigned, not system-assigned? Application Gateway’s Key Vault certificate integration specifically requires a user-assigned managed identity; a system-assigned identity is not supported for this integration. The UAMI is also reusable across gateways and survives gateway re-creation, so one grant on the vault serves the whole fleet.

4. What is the difference between a versioned and an unversioned secret ID, and why does it matter for rotation? A versioned ID pins one exact cert version and freezes the gateway on it; an unversioned ID makes the periodic poll resolve to the current version, so renewals auto-rotate. Binding versioned is the classic bug: the cert renews under a new version, the pinned old one expires, and the gateway serves a dead cert. az keyvault secret show --query id returns the versioned URL, so strip the trailing segment.

5. A gateway serves an expired certificate although the central vault holds a freshly renewed one. Diagnose it. The binding is versioned — frozen on the now-expired version. Confirm with ssl-cert show --query keyVaultSecretId; a trailing GUID means versioned. Fix by re-binding the unversioned id (${RAW%/*}) plus a CI guardrail against versioned ids.

6. The cert shows state “Unknown” versus “Failed” — what’s the difference? Unknown means the gateway authenticated but couldn’t fetch — a network/firewall problem (KV default-deny with no open path, or DNS resolving the vault’s public IP behind a PE). Failed more often means authorization or the secret (no role, disabled secret, wrong URL). Unknown → fix the network path; Failed → fix the role/secret.

7. You add a Private Endpoint but the cert still won’t load. Most likely cause? DNS. The gateway’s VNet isn’t resolving privatelink.vaultcore.azure.net, so the vault hostname resolves to its public IP and the default-deny firewall refuses it. Confirm with nslookup from the gateway’s VNet (must return the private 10.x). Fix by linking the privatelink zone to the VNet (or fronting it with the Private Resolver).

8. How do you grant the cross-subscription role in Terraform, given the gateway is in one subscription and the vault in another? Configure two azurerm providers — default for Connectivity, an alias for the Identity subscription — and create the azurerm_role_assignment with provider = azurerm.identity, scoped to the vault’s resource ID, granting the UAMI’s principal_id the Key Vault Secrets User role. Add depends_on so the gateway provisions after the grant.

9. Why might re-creating a vault or certificate with the same name fail, and how do you handle it? Soft-delete reserves the name for the retention window (and purge protection may block early purge). A delete-then-recreate collides with the soft-deleted shadow (Conflict). Recover it (az keyvault recover) or, if purge is allowed and you want a clean slate, purge it — but in CI prefer versioning in place so you never delete-recreate.

10. How fast does a renewal reach the gateway, and how do you avoid a rotation-induced outage? The gateway polls Key Vault roughly every four hours, so rotation is not instant. Avoid outages by renewing with generous margin (30+ days before expiry) so the new version is live well before the old one’s notAfter; if you need it immediately, force a re-pull with ssl-cert update. Never renew on the expiry day.

11. What’s the least-privilege scope for the role assignment, and what’s the trade-off of scoping wider? Scope it to the vault. Scoping to the resource group or subscription grants the gateway read on every secret in scope — over-broad on a central secrets store. Scoping to a single secret via ABAC conditions is possible but fiddly and breaks on rename; if you want hard isolation, use a dedicated certs vault so vault-scope and least privilege coincide.

12. The same central cert needs to serve Front Door and API Management too. Does the pattern change? No — the identity-plus-secret-GET pattern generalizes. Each consumer authenticates with its own (or a shared) user-assigned identity granted Key Vault Secrets User on the central vault and references the cert by its unversioned secret ID. The cross-subscription mechanics are identical; only the consumer resource changes.

These map to AZ-700 (Network Engineer)design and implement Application Gateway, secure access to Key Vault, private endpoints — and AZ-500 (Security Engineer)manage Key Vault, RBAC, managed identities, certificate lifecycle. The landing-zone design framing touches AZ-305 (Solutions Architect). A compact cert-mapping for revision:

Question theme Primary cert Objective area
AppGW listener TLS + KV cert binding AZ-700 Design & implement Application Gateway
Cross-subscription RBAC, UAMI AZ-500 / AZ-104 Manage identities & access; RBAC
/secrets/ vs /certificates/, least privilege AZ-500 Implement & manage Key Vault
Private Endpoint + private DNS to KV AZ-700 Secure access with private endpoints
Auto-rotation, cert lifecycle, soft-delete AZ-500 Configure & manage certificate lifecycle
Centralization, blast radius, separation of duties AZ-305 Design identity, governance & monitoring

Quick check

  1. Application Gateway reads a Key Vault certificate through which of its three object representations, and what’s the corresponding least-privilege role?
  2. You bind the listener’s SSL cert and a colleague worries auto-rotation will break. What single property of the bound secret ID do you check, and what must it look like?
  3. The cert shows state Unknown on the gateway even though the UAMI clearly has Key Vault Secrets User. Name the two most likely causes and the one command that distinguishes them.
  4. In Terraform, the gateway is in the Connectivity subscription and the vault in Identity. What’s the mechanism that lets you create the role assignment in the vault’s subscription?
  5. A central wildcard renewed on schedule, yet the gateway serves an expired cert. What’s the root cause and the fix?

Answers

  1. The secret representation (/secrets/<name>), because it contains the full cert including the private key needed to terminate TLS. The least-privilege role is Key Vault Secrets User (GET on secrets) — not a certificates role.
  2. Check that the keyVaultSecretId is unversioned — it must end in /secrets/<name> with no trailing version GUID. If it ends in a GUID it’s pinned and auto-rotation is broken; re-bind with ${RAW%/*}.
  3. (a) The KV firewall (default-deny) is blocking the fetch, or (b) the gateway’s VNet resolves the vault’s public IP despite a Private Endpoint (DNS). Distinguish with nslookup <vault>.vault.azure.net from the gateway’s VNet — a public IP means it’s the DNS/firewall path, not RBAC. (Unknown ≠ Failed: Unknown is “authenticated but can’t reach.”)
  4. Configure a second azurerm provider with an alias pointed at the Identity subscription and create the azurerm_role_assignment with provider = azurerm.identity, scoped to the vault’s resource ID. (Bicep equivalent: a module with scope: resourceGroup(identitySubId, identityKvRg).)
  5. The listener was bound by the versioned secret ID, freezing the gateway on the old (now-expired) version; the renewal landed under a new version the gateway never polls. Fix by re-binding the unversioned secret ID and add a CI guardrail to prevent versioned bindings.

Glossary

Next steps

You can now bind a gateway to a central, cross-subscription certificate correctly — unversioned, least-privilege, private network path, with an expiry alarm. Build outward:

Application GatewayKey VaultManaged IdentityCross-SubscriptionTLS CertificatesCert RotationPrivate EndpointLanding Zone
Need this built for real?

Vinod is a Senior Cloud Architect (22+ yrs) — available for Azure / AWS / GCP architecture, landing zones, and migrations.

Work with me

Comments

Keep Reading