Key Vault 403 Forbidden: Untangling Firewall, RBAC, Purge Protection, and Soft-Delete Recovery

You run az keyvault secret show, or your app boots, or a pipeline tries to read a connection string — and you get 403 Forbidden: “…does not have secrets get permission on key vault” or “Client address is not authorized and caller is not a trusted service”. You can see the vault, the secret is clearly there — yet the value is denied. This is the most common Azure Key Vault support call there is, confusing for one reason: a 403 from Key Vault is two completely different failures wearing the same number. Either the network said no (the vault’s firewall blocked where you’re calling from), or authorization said no (your identity lacks permission to do what you asked). The fix for one does nothing for the other, so the whole game is telling them apart in the first ninety seconds.

This is that playbook, written for someone newer to Azure. You will learn the two-gate model every Key Vault request passes through — network gate (firewall / private endpoint / trusted services) then authorization gate (RBAC or access policies, never both) — and how to pin any 403 to one gate using commands that tell the truth: az keyvault show, az role assignment list, az webapp identity show, and the vault’s diagnostic logs. Every diagnosis comes with the exact confirm step and fix, in az and Bicep. And because newcomers hit a second trap — you “deleted” a vault and now can’t reuse the name — we cover soft-delete and purge protection: what they retain, how to recover, and the one setting you can never undo. By the end the 403 is a short, mechanical diagnosis, not an afternoon of clicking blades.

What problem this solves

Key Vault keeps secrets out of your code — but the moment it does its job it can deny you, and the denial is deliberately vague at the edge (a 403 must not reveal whether the secret exists or who has access, or an attacker learns from it). The information you need is logged, not in the error string, so without knowing which gate to inspect you waste the incident granting permissions to fix a firewall, or opening the firewall to fix permissions. What breaks is concrete: an app fine in dev refuses its connection string in prod, restart-loops, and on-call “fixes” it by pasting the secret back into plaintext app settings — undoing the whole point. A pipeline 403s because someone enabled the firewall for compliance and forgot the build agents. A developer can’t recreate a deleted vault’s name because it’s soft-deleted, not gone. Each is a five-minute fix if you know where to look, a multi-hour mystery if you don’t.

It bites hardest on teams adopting managed identities for the first time (the identity exists but has no role), teams that flipped on RBAC while people still relied on access policies, anyone who enabled the firewall without allow-listing real callers, and newcomers who don’t yet realise soft-delete can’t be turned off.

Learning objectives

By the end of this article you can:

• Explain the two-gate model — network (firewall/PE/trusted services) then authorization (RBAC or access policies) — and classify any 403 as a network or authorization denial.
• Tell a network 403 from an authorization 403 by the error string and a single az command, and stop fixing the wrong gate.
• Fix an authorization 403: missing data role, wrong scope, RBAC-vs-policy mismatch, an identity never enabled, or post-grant replication delay.
• Fix a network 403: firewall default-deny, an IP not allow-listed, trusted services off, or a private-endpoint/DNS misconfiguration.
• Pick the right data-plane role (Secrets User vs Officer vs Crypto User) and grant it at the right scope with az and Bicep.
• Recover from the soft-delete trap, reuse a “taken” name, and know what purge protection prevents — including the one setting you can never reverse.
• Read the diagnostic logs to prove who was denied, from where, and why.

Prerequisites & where this fits

You should know the basics: a vault holds secrets, keys and certificates, read and written over a data plane (https://<vault-name>.vault.azure.net) that is separate from the ARM control plane that creates and configures the vault — a split that is the single most important idea here. You should run az in Cloud Shell, read JSON, and have skimmed Azure Key Vault: Secrets, Keys and Certificates Done Right.

This sits in the Observability & Troubleshooting track. If managed identities are new, read Managed Identities Demystified: System vs User-Assigned and When to Use Each first — half of all production Key Vault 403s are an app identity that holds no role. The model choice has its own guide in Key Vault RBAC vs Access Policies: Choosing and Migrating the Permission Model. The same two-failures pattern appears on storage — see Fixing Azure Storage 403 Errors: Firewalls, Private Endpoints, RBAC & SAS.

A map of who owns each gate, so you call the right person fast:

Core concepts

Four mental models make every later diagnosis obvious.

Control plane and data plane are different doors. The control plane (Azure Resource Manager) creates the vault, sets its firewall and assigns roles, governed by roles like Contributor. The data plane is the vault’s endpoint (https://<vault>.vault.azure.net) where secrets/keys/certs are read and written. The trap that catches every newcomer: being Contributor (or even Owner) lets you manage the vault but does not let you read a secret value — that needs a separate data-plane permission. “I’m Owner and still get 403” is this split working as designed.

Every data-plane request passes two gates, in order. First the network gate: if the firewall is on, Key Vault checks where the call came from (allow-listed IP, allowed VNet/private endpoint, or a “trusted Microsoft service”) and refuses non-matches before authentication even matters. Then the authorization gate checks who you are and what you asked. Each gate’s 403 carries a different error string — reading which gate spoke is the whole skill.

Authorization is RBAC or access policies — exactly one is active, set by enableRbacAuthorization (true → data roles count, policies ignored; false → policies count, data roles ignored). The most common authorization 403 is a mismatch — granting in the model the vault isn’t using — so always confirm the active model first.

Nothing is deleted immediately — soft-delete is always on. A deleted secret/key/cert/vault enters a soft-deleted state (7–90 days, default 90): hidden from listing but holding its name and recoverable, and it cannot be disabled on recent vaults. Optional purge protection also blocks an early purge until retention expires — and once enabled, cannot be turned off.

The vocabulary in one table

The moving parts side by side (the glossary at the end repeats these for lookup):

One command that frames every diagnosis

A 403 failed at exactly one gate. Before you even read the error string, one command snapshots both gates’ state — make this your first move every time:

# Snapshot both gates (and the recovery flags) for a vault in one call
az keyvault show --name myvault --resource-group rg-app \
  --query "{rbac:properties.enableRbacAuthorization, publicAccess:properties.publicNetworkAccess, defaultAction:properties.networkAcls.defaultAction, purgeProtection:properties.enablePurgeProtection}" -o jsonc

defaultAction: Deny means a firewall is in play (the network gate can bite); rbac:true means look for role assignments, rbac:false means look at access policies. That one output frames the diagnosis.

Anatomy of an authorization 403

An authorization 403 means you cleared the network gate but your identity can’t do what you asked — for a beginner, the more common of the two. Five causes, all looking like “I granted access and it still 403s,” cause by cause with the confirming command.

Cause 1 & 2 — The identity has no data-plane permission (the Owner trap)

The number-one Key Vault 403: the caller (an app’s managed identity, a CI/CD service principal, or a developer) has no data-plane grant. The cruel variant is being Owner or Contributor — control-plane roles that let you rename, re-firewall, even delete the vault, but do not include secrets/get, gated separately so a broad Contributor can’t silently read every secret.

Confirm. Get the caller’s identity and list what it holds on the vault — an empty result (or only Contributor/Reader) is the smoking gun. Use the RBAC line on an RBAC-mode vault, the policy line otherwise:

PRINCIPAL=$(az webapp identity show --name app-shop-prod -g rg-app --query principalId -o tsv)
VAULT_ID=$(az keyvault show --name myvault -g rg-app --query id -o tsv)
az role assignment list --assignee "$PRINCIPAL" --scope "$VAULT_ID" -o table   # RBAC mode
az keyvault show --name myvault -g rg-app \
  --query "properties.accessPolicies[?objectId=='$PRINCIPAL']" -o jsonc          # policy mode

Fix (RBAC mode). Grant the least-privilege data role — Key Vault Secrets User to read secrets — scoped to the vault, not the subscription:

az role assignment create --assignee "$PRINCIPAL" --role "Key Vault Secrets User" --scope "$VAULT_ID"

// Data-plane role assignment, scoped to the vault, for the app's identity
resource secretsUser 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  name: guid(vault.id, app.id, 'kv-secrets-user')
  scope: vault
  properties: {
    // Key Vault Secrets User — built-in role definition GUID
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', '4633458b-17de-408a-b874-0445c86b69e6')
    principalId: app.identity.principalId
    principalType: 'ServicePrincipal'
  }
}

Fix (access-policy mode). Add a policy with just the operations needed: az keyvault set-policy --name myvault --object-id "$PRINCIPAL" --secret-permissions get list.

Pick the narrowest role that does the job. The data-plane roles you’ll reach for most:

Note the Reader trap: it lists secret names/properties but not values — newcomers grant Reader, see the secret appear, and are baffled by the 403 on the value. Reading values needs Secrets User. (Keys and certs have their own Officer roles too.)

Cause 3 — RBAC-vs-access-policy mismatch

You granted access, you can see the grant, and it still 403s — because you granted in the wrong model. On an enableRbacAuthorization = true vault, access policies are ignored; on a false vault, RBAC data roles are ignored. Fix it by granting in the active model — or migrating the vault to RBAC:

az keyvault show   --name myvault -g rg-app --query "properties.enableRbacAuthorization" -o tsv  # true/false
az keyvault update --name myvault -g rg-app --enable-rbac-authorization true                    # migrate

Migrate deliberately, not mid-incident: flipping a busy vault to RBAC instantly invalidates every access policy, so anything relying on one 403s until you recreate it as a role. The two models side by side:

Causes 4 & 5 — Wrong scope, and replication delay

Two more “still 403s” cases. Wrong scope: RBAC roles are scoped, so a Secrets User assignment on vault A (or an RG, or the subscription) does nothing for vault B — az role assignment list --assignee "$PRINCIPAL" --all shows the scope, which must contain this vault’s resource ID; re-create it on the vault for least privilege. Replication delay: a fresh grant is eventually consistent (commonly under 10 minutes), and a token issued before it won’t carry the permission until refreshed — wait and retry, or restart the app to refresh its token. If it still fails after ~15 minutes it isn’t replication; recheck model and scope.

Anatomy of a network 403

A network 403 means the request never reached the authorization gate — the firewall refused your source. This ambushes teams after a compliance hardening: the vault worked yesterday, someone restricted public access, half the callers are now blocked. Four causes:

Cause 1 — Firewall default-deny and your IP isn’t allow-listed

With the firewall on, defaultAction is Deny and only listed IPs/VNets pass. The subtlety: the IP that matters is your egress IP as Azure sees it — often not the address in your OS settings, since NAT/proxies/home routers rewrite it. The error string even names the blocked address — “Client address 203.0.113.5 is not authorized…”. Compare it to the allow-list, then add it:

az keyvault show --name myvault -g rg-app --query "properties.networkAcls" -o jsonc  # defaultAction + ipRules
curl -s https://api.ipify.org; echo                                                  # your real egress IP
az keyvault network-rule add --name myvault -g rg-app --ip-address 203.0.113.5/32     # fix

Cause 2 — “Allow trusted Microsoft services” is off

Many Azure services call Key Vault for you — App Service resolving a reference, storage using a customer-managed key, Backup, ARM deployments. With the firewall on these are blocked unless the bypass allows trusted services. Newcomers add their office IP and then App Service 403s, because it isn’t an IP you can list — it’s a platform service needing the exception. Enable it:

az keyvault show   --name myvault -g rg-app --query "properties.networkAcls.bypass" -o tsv  # None = blocked
az keyvault update --name myvault -g rg-app --bypass AzureServices --default-action Deny    # fix

This covers Microsoft-owned registered services — not a blanket “allow all Azure.” For your own apps, prefer a private endpoint over the bypass. What the firewall checks:

Causes 3 & 4 — Private endpoint and the DNS gotcha

The cleanest production posture is to disable public access (publicNetworkAccess = Disabled) and reach the vault over a private endpoint — a private IP in your VNet. But a private endpoint without matching private DNS is the classic half-configured state: the endpoint exists, yet the app resolves the vault to its public IP and 403s because public access is off. Confirm from a host in the VNet:

nslookup myvault.vault.azure.net
# Healthy: a 10.x private IP via privatelink.vaultcore.azure.net
# Broken:  a public IP -> hits the (disabled) public endpoint -> 403

The fix: link the privatelink.vaultcore.azure.net private DNS zone to the VNet and confirm the endpoint’s A record. Full mechanics in Azure Private Endpoint vs Service Endpoint: Secure PaaS Access and Azure Private Link and Private DNS: Keeping PaaS Off the Public Internet.

The error-code & message reference

The exact 403 wording classifies the gate; here it is alongside the non-403 codes you’ll also meet around Key Vault, and what they really mean:

The rule that saves the most time: a network 403 (“Client address…”) is never fixed by granting a role, and an authorization 403 (“…does not have … permission”) is never fixed by opening the firewall.

Soft-delete and purge protection: the recovery model

The second newcomer trap isn’t firewall or RBAC — it’s deletion. You delete a vault, try to recreate the name, and Azure refuses with “already in use” though nothing is visible. Soft-delete retains the deleted vault/secret/key/cert for 7–90 days (default 90), holding its name and recoverable to its prior state — it cannot be disabled on recent vaults, and it saves you from a fat-fingered delete. Purge protection then blocks even an early purge (permanent delete) for everyone, even owners, until retention expires — compliance- and ransomware-grade — but cannot be turned off once enabled, so on a throwaway vault you’ve committed to that name for up to 90 days. Side by side:

Recovering a soft-deleted vault or secret

List what’s holding the name, then recover (or, only with no purge protection, purge to release it — irreversible):

# Vaults: list soft-deleted (they hold the names), then recover or purge
az keyvault list-deleted --query "[].{name:name, scheduledPurge:properties.scheduledPurgeDate}" -o table
az keyvault recover --name myvault --location eastus
az keyvault purge   --name myvault --location eastus   # PERMANENT; only if purge protection is OFF

# Secrets inside an existing vault: list soft-deleted, then recover
az keyvault secret list-deleted --vault-name myvault -o table
az keyvault secret recover --vault-name myvault --name db-conn

Purging needs the purge permission (Key Vault Administrator) and itself 403s if you lack it or purge protection blocks it. Set protections in IaC so they’re never accidental:

resource vault 'Microsoft.KeyVault/vaults@2023-07-01' = {
  name: 'myvault'
  location: location
  properties: {
    tenantId: subscription().tenantId
    sku: { family: 'A', name: 'standard' }
    enableRbacAuthorization: true
    enableSoftDelete: true               // always on; recent API versions ignore 'false'
    softDeleteRetentionInDays: 90        // 7–90
    enablePurgeProtection: true          // IRREVERSIBLE once deployed
    networkAcls: {                       // the firewall, as code
      defaultAction: 'Deny'
      bypass: 'AzureServices'            // trusted services (App Service references, Backup, ARM)
      ipRules: [ { value: '203.0.113.5/32' } ]
    }
  }
}

A blunt warning: don’t set enablePurgeProtection: true on scratch/CI vaults — you’ll fill your subscription with un-purgeable, name-locked vaults for up to 90 days each.

Architecture at a glance

Picture the request as a corridor with two doors in series. A caller — App Service or Function (via its managed identity), a CI/CD service principal, or you at a terminal — sends a request to https://<vault>.vault.azure.net. Door one, the network gate: if the firewall is on, the vault checks where the call came from (egress IP, allowed VNet, private endpoint, or the trusted services exception) and, failing, returns “Client address is not authorized” before your identity matters. Door two, the authorization gate: the vault checks your identity and operation against the one active model — RBAC or access policies — and, failing, returns “does not have secrets get permission.” Only a request clearing both doors succeeds.

Off to the side, not on the request path, sits the recovery model: every delete drops the object into a soft-deleted holding area (7–90 days) where it still owns its name and is recoverable, and purge protection stops even an admin erasing it early. Keep these three — network gate, authorization gate, recovery model — separate and you’ll never grant a role to fix a firewall.

Real-world scenario

Lumio Retail, a mid-size e-commerce shop, runs its checkout API on Azure App Service, reading its DB connection string and a payment-gateway key from Key Vault via Key Vault references, resolved by the app’s system-assigned managed identity. It worked for months. Then security ran a CIS hardening pass before a PCI audit and enabled the vault firewall (defaultAction: Deny) with the office IP allow-listed — but left bypass at None.

At the next deploy the checkout API began restart-looping: the connection-string setting resolved to empty, the app threw on startup, recycled, threw again — a crash loop with no application exception naming the cause. On-call took the classic wrong turn: assume permissions, start adding RBAC roles. Nothing changed, because the identity already had Key Vault Secrets User — the authorization gate was fine. They were fixing door two while door one was shut.

The breakthrough came from the Environment variables blade (the reference showed a red error) and the vault’s diagnostic logs, which recorded the denied request with ResultSignature: Forbidden and a caller that was not an office IP — it was App Service’s platform identity. The tell: not an IP they could list, but a trusted Microsoft service blocked because bypass was None. The fix was one control-plane change:

az keyvault update --name kv-lumio-prod --resource-group rg-lumio-prod \
  --bypass AzureServices --default-action Deny

The team then went further — a private endpoint with publicNetworkAccess: Disabled, so the API reaches the vault over a private IP needing no allow-list and the internet can’t touch it. The runbook lesson, in bold: a Key Vault 403 after a network change is a network 403 — read the message and check the firewall before touching a single role. The second time it happened (different vault, same pattern): four minutes.

Advantages and disadvantages

Key Vault’s two-gate, soft-delete-by-default design is what makes it secure — and what makes 403s and “name taken” errors so common:

Every disadvantage is the security design doing its job: the friction (two gates, locked names, irreversible flags) is exactly what stops a Contributor reading production secrets or a delete wiping them out — which is why you keep purge protection off on throwaway vaults and on in production.

Hands-on lab

This free-tier-friendly lab reproduces both 403 classes on purpose, then fixes each. You need Cloud Shell (az). A standard vault has no hourly charge (you pay per-operation, fractions of a paisa), so this costs essentially nothing. Use a globally-unique vault name.

1. Create an RBAC-mode vault and a secret.

RG=rg-kv-lab
LOC=eastus
VAULT=kvlab$RANDOM            # must be globally unique
az group create -n $RG -l $LOC
az keyvault create -n $VAULT -g $RG -l $LOC --enable-rbac-authorization true
az keyvault secret set --vault-name $VAULT --name demo --value "hello"   # may 403!

If you are only Owner, that secret set may 403 — the Owner-trap, live (in RBAC mode the creator isn’t automatically a data-plane admin). Grant yourself a data role and retry:

ME=$(az ad signed-in-user show --query id -o tsv)
VID=$(az keyvault show -n $VAULT -g $RG --query id -o tsv)
az role assignment create --assignee "$ME" --role "Key Vault Administrator" --scope "$VID"
# Wait ~1–2 min for replication, then:
az keyvault secret set --vault-name $VAULT --name demo --value "hello"
az keyvault secret show --vault-name $VAULT --name demo --query value -o tsv   # -> hello

2. Reproduce an authorization 403 with a service principal that has only a control-plane role:

SP=$(az ad sp create-for-rbac --name sp-kv-lab --skip-assignment --query appId -o tsv)
az role assignment create --assignee "$SP" --role "Reader" --scope "$VID"   # control-plane only
# As that SP, 'secret show' -> 403 "does not have secrets get permission"
az role assignment create --assignee "$SP" --role "Key Vault Secrets User" --scope "$VID"  # fix

3. Reproduce a network 403 — turn on the firewall and omit your IP, then allow-list it:

az keyvault update -n $VAULT -g $RG --default-action Deny --bypass AzureServices
az keyvault secret show --vault-name $VAULT --name demo   # -> 403 "Client address is not authorized"
az keyvault network-rule add -n $VAULT -g $RG --ip-address $(curl -s https://api.ipify.org)/32  # fix

Note step 3’s wording differs from step 2 — same status, different gate.

4. Feel the soft-delete trap, then teardown (this vault has no purge protection, so the name can be released):

az keyvault delete -n $VAULT -g $RG
az keyvault create -n $VAULT -g $RG -l $LOC   # -> 409 "already in use" (soft-deleted!)
az keyvault recover -n $VAULT -l $LOC          # brings it back, secret intact
az keyvault delete -n $VAULT -g $RG && az keyvault purge -n $VAULT -l $LOC   # permanent
az ad sp delete --id "$SP"; az group delete -n $RG --yes --no-wait

You have now produced and fixed both 403 classes and walked the soft-delete lifecycle — the muscle memory the next incident needs.

Common mistakes & troubleshooting

This is the centerpiece. Find your symptom, confirm with the exact command/path, apply the fix — causes span the simple (no role) to the subtle (DNS, replication, purge protection):

The one clarification the table can’t carry: rows 1–4 all look identical (“I granted access and it still 403s”) but each differs by one check — work them in order: (1) is there a data role at all? (2) is it a data role, not Owner/Contributor? (3) is the vault in the model you granted in? (4) is the role scoped to this vault? The first “no” is your cause.

To diagnose any of this with certainty, turn on diagnostic logs — they record every allowed and denied operation with caller, IP, operation and result. Do it before the next incident:

# Send Key Vault audit + all logs to a Log Analytics workspace
WS=$(az monitor log-analytics workspace show -g rg-obs -n law-shared --query id -o tsv)
az monitor diagnostic-settings create --name kv-audit \
  --resource "$VAULT_ID" \
  --workspace "$WS" \
  --logs '[{"categoryGroup":"audit","enabled":true},{"categoryGroup":"allLogs","enabled":true}]'

Then a denied request is one query away:

// Every denied Key Vault operation in the last hour: who, from where, what
AzureDiagnostics
| where ResourceType == "VAULTS" and ResultSignature == "Forbidden"
| where TimeGenerated > ago(1h)
| project TimeGenerated, OperationName, CallerIPAddress, identity_claim_upn_s, requestUri_s
| order by TimeGenerated desc

An unexpected CallerIPAddress means a network block; a right caller with a Forbidden permission message means an authorization gap. The log removes the last of the guessing.

Best practices

• Confirm the model first. Run az keyvault show --query properties.enableRbacAuthorization before anything — half of “I granted it and it still 403s” is the wrong model. Prefer RBAC on new vaults.
• Grant data-plane least privilege, scoped to the vault. Read-only apps get Key Vault Secrets User on the vault, never Contributor or subscription-wide. Keys need a key role, certs a cert role — separate.
• Never confuse control-plane and data-plane — Owner/Contributor manage the vault but don’t read values; and use managed identities, enabling the identity before you grant the role.
• When the firewall is on, allow-list every real caller — your egress IP/CIDR and bypass = AzureServices for platform services — or, better, a private endpoint + publicNetworkAccess = Disabled with the privatelink.vaultcore.azure.net zone.
• Turn on diagnostic logs from day one so every denial names its caller, IP, operation and gate. Route to Log Analytics.
• Enable purge protection only on production — it’s irreversible and locks names, so keep it off on scratch/CI vaults.
• Manage access and networking as code (role assignments, networkAcls, protection flags in Bicep, reviewed in PRs) — a hand-clicked firewall change is how the “App Service suddenly 403s” incident starts.
• Document the egress-IP gotcha — “add your IP” means your public egress IP (curl https://api.ipify.org), not the OS-settings address; this wastes the most beginner time.

Security notes

• Least privilege by default. Narrowest data role: Secrets User to read, Secrets Officer only where the workload writes, Administrator for break-glass. Reader sees metadata only — right for auditors, wrong for reading values.
• Managed identity over stored credentials — no client secret to leak. Grant it the data role and nothing else.
• Network-isolate production vaults with default-deny plus a private endpoint; reserve the trusted-services bypass for Microsoft platform callers, not as a substitute for private networking.
• Log and alert on denials — a spike in Forbidden is a misconfiguration or an attacker probing, and diagnostic logs let you tell. Pair with Azure Monitor and Application Insights: Full-Stack Observability.
• Purge protection for anything you can’t afford to lose — defends against accidental deletion and malicious early purge (a real ransomware tactic), at the cost of irreversibility.
• Don’t paste secrets back into config to “unblock.” Reverting to plaintext app settings is the most damaging response to a 403 — fix the gate, never undo the isolation.
• Separate duties via RBAC + PIM — because data roles are Azure RBAC, Key Vault Administrator can be PIM-eligible (just-in-time), which access policies can’t do.

Cost & sizing

Key Vault is cheap to run and priced by use — the cost story is the right tier and not letting protections create waste:

• Standard vs Premium, priced per operation. A standard vault has no standing charge — you pay per operation at a few paise per 10,000, a rounding error. Premium adds HSM-backed keys (FIPS 140-2 Level 2), billed per HSM key per month; choose it only for hardware-protected key material, not secrets. The bill scales with how often you read, so cache (App Service references already do).
• Soft-delete and purge protection are free — they cost names and time, not money: a purge-protected vault locks its name for up to 90 days, cluttering the subscription if used on scratch vaults.
• Private endpoints and diagnostic logs are the only real charges — a PE bills a small hourly + per-GB rate (worth it for production isolation), and logs bill as Log Analytics ingestion per GB (low for Key Vault, far cheaper than an undiagnosable incident).

A rough monthly picture for a small production app: a standard vault is well under ₹100/month in operations; add a private endpoint (~₹500–900) and modest Log Analytics (~₹100–500) and you’re still under ~₹1,500 all-in — the fix for a 403 is almost never “spend more,” it’s a role, a firewall rule, or a DNS link.

Interview & exam questions

1. A user gets 403 reading a secret, but App Service logs show the request reaching the vault. What two failures could this be, and how do you tell them apart? Either a network 403 (firewall blocked the source) or an authorization 403 (identity lacks the permission). The message tells you: “Client address is not authorized…” is the network gate; “does not have secrets get permission” is authorization. Confirm with az keyvault show and az role assignment list.

2. An engineer is Owner on the vault and still gets 403 reading a secret value. Why? Owner is a control-plane role — it manages the vault but does not include the data-plane secrets/get; Key Vault gates data separately so broad Owners can’t silently read secrets. Fix with a data role like Key Vault Secrets User, scoped to the vault.

3. Difference between RBAC mode and access-policy mode, and how does it cause a 403? A vault uses one model (enableRbacAuthorization): RBAC mode honours role assignments and ignores access policies; policy mode does the reverse. The classic 403 is granting in the wrong model, so the grant silently does nothing.

4. A vault firewall is on and an App Service Key Vault reference suddenly 403s. Likely cause and fix? The trusted services bypass is off (bypass = None), so App Service’s platform-side resolution is blocked despite IP rules — and it isn’t an IP you can list. Set bypass = AzureServices, or better, give the vault a private endpoint.

5. You add a private endpoint and disable public access, but the app still 403s. Why? DNS. With public access off, the app must resolve the vault to its private IP via the privatelink.vaultcore.azure.net zone; if that zone isn’t linked to the VNet, the name resolves to the public IP and the disabled endpoint refuses it. Link the zone and confirm the A record.

6. What is soft-delete, can it be turned off, and how do you recover a deleted secret? Soft-delete retains deleted objects for a 7–90 day window (default 90) instead of destroying them; it is always on and cannot be disabled on recent vaults. Recover with az keyvault recover (vault) or az keyvault secret recover (secret) within the window.

7. What does purge protection do, and the one thing to remember? It blocks the purge (permanent early delete) until retention expires, even for admins — compliance-grade, ransomware-resistant retention. The one thing: it cannot be turned off once enabled, so never set it on throwaway vaults (it locks their names up to 90 days).

8. You try to create a vault and get “name is already in use,” but nothing is visible. What happened, and your options? A soft-deleted vault holds the name during retention. az keyvault list-deleted finds it; then recover it to get it back, or purge it to release the name — purge only works if purge protection is off and you hold purge permission.

These map to AZ-204 (manage keys, secrets and certificates with Key Vault) and AZ-500 (access policies vs RBAC, network restrictions, recovery); the networking angle touches AZ-104/AZ-700. A compact mapping for revision:

Quick check

1. You get “Client address is not authorized and caller is not a trusted service.” Which gate refused you, and is granting an RBAC role going to help?
2. You’re Owner on the vault and still get 403 reading a secret value. Why, and what role fixes it?
3. You granted Key Vault Secrets User to the app and it still 403s. Name the first vault property you check to explain it.
4. You delete a test vault and can’t recreate it with the same name, with nothing visible in the portal. What happened, and which command shows it?
5. True or false: purge protection can be turned off later if it gets in your way.

Answers

1. The network gate — the firewall blocked your source IP. An RBAC role does nothing; fix the network (add the egress IP, enable trusted services, or use a private endpoint).
2. Owner is control-plane and excludes the data-plane secrets/get. Add a data role — Key Vault Secrets User (read) or Administrator (full) — scoped to the vault.
3. enableRbacAuthorization. False → access-policy mode, your RBAC role is ignored (add a policy or migrate to RBAC). True → re-check the role’s scope.
4. The vault is soft-deleted, holding its name for 7–90 days. az keyvault list-deleted shows it; then az keyvault recover, or az keyvault purge to release the name (only if purge protection is off).
5. False. Purge protection is irreversible — which is why you keep it off scratch vaults and reserve it for production.

Glossary

• Control plane — ARM operations that create/configure the vault (firewall, role assignments); governed by roles like Contributor. Does not grant data access.
• Data plane — the vault’s endpoint (https://<vault>.vault.azure.net) where secrets/keys/certs are read and written; where the 403 is thrown.
• Network gate / authorization gate — the two checks every request passes: first the firewall/PE/trusted-services check on your source, then the RBAC-or-policy check on your identity.
• RBAC mode / access-policy mode — enableRbacAuthorization true (honours data roles, ignores policies) or false (honours policies, ignores data roles).
• Key Vault Secrets User — data role granting read (get/list) on secret values; least privilege for apps that read secrets.
• Key Vault Reader — grants read of metadata only (names/properties), not values — a common 403 when mistaken for Secrets User.
• Managed identity — a system- or user-assigned identity an app uses to authenticate to Key Vault with no stored credential.
• Trusted Microsoft services (bypass) — firewall exception (bypass = AzureServices) letting registered Azure services (App Service references, Backup, ARM) reach the vault despite default-deny.
• Private endpoint — a private IP for the vault in your VNet; with public access disabled, the supported path in — and it needs the privatelink.vaultcore.azure.net DNS zone.
• Soft-delete — always-on retention of deleted objects for 7–90 days (default 90), holding the name and recoverable; cannot be disabled.
• Purge protection — optional, irreversible setting blocking permanent early delete (purge) until retention expires, even for admins.
• Key Vault reference — an app setting @Microsoft.KeyVault(SecretUri=…) resolved at runtime via the app’s managed identity; fails (empty value, crash loop) if any gate refuses it.
• Diagnostic logs — the audit stream recording every operation (caller, IP, operation, result) to Log Analytics; the definitive source for classifying a 403.

Next steps

You can now classify any Key Vault 403 to a gate and fix it, and recover from the soft-delete trap. Build outward:

• Next: Azure Key Vault: Secrets, Keys and Certificates Done Right — the foundations of what the vault stores and how to use it well.
• Related: Key Vault RBAC vs Access Policies: Choosing and Migrating the Permission Model — pick and migrate the authorization model deliberately, not mid-incident.
• Related: Managed Identities Demystified: System vs User-Assigned and When to Use Each — get the identity right so it has something to authorize.
• Related: Fixing Azure Storage 403 Errors: Firewalls, Private Endpoints, RBAC & SAS — the same two-gate pattern on storage.
• Related: Azure Private Endpoint vs Service Endpoint: Secure PaaS Access — wire the private path (and DNS) that removes IP-allow-list churn for good.