Networking Azure

Private Endpoints and DNS at Scale: Centralized Private DNS Zone Architecture

A single private endpoint is a five-minute job. A fleet of them — hundreds of endpoints across dozens of spokes, each needing an A record that resolves consistently from every VNet, from on-prem across ExpressRoute, and from a peered tenant — is a governance problem dressed up as a networking one. The core mechanic is deceptively small: the name your application uses never changes when you add a private endpoint. Your SDK, your connection string, and the resource’s TLS certificate all reference the public FQDNstappdata.blob.core.windows.net, kv-app.vault.azure.net, sql-orders.database.windows.net. A private endpoint projects a NIC into your VNet and hands the target PaaS resource a private IP, but it does nothing to that public name. The entire discipline of “Private Link DNS at scale” is making that public FQDN resolve to the private IP for every client that must reach it, and only for those clients — without hand-writing a single A record and without one spoke team silently shipping an endpoint that resolves to a public IP.

This article is the Advanced-level, end-to-end treatment. We start from the DNS chain Azure half-builds (the public CNAME into the privatelink.* alias, and the A record you host), then build the two topologies — a zone per spoke, or one canonical set in the hub — and make the centralized model self-enforcing with Azure Policy DeployIfNotExists. From there we go deep on the three ways to answer a private query for non-VNet clients — Azure DNS Private Resolver (inbound/outbound endpoints, forwarding rulesets), custom DNS forwarder VMs, and directly-linked Private DNS Zones — and when each is right. We cover on-prem resolution over ExpressRoute private peering, conditional forwarding in both directions, the per-service zone-name catalogue (the biggest source of silent failures), and the override traps that make resolution “work on some clients and not others.” Every decision comes with a table, every operation with az and Bicep, closing with a scaled reference design, a lab, and a troubleshooting playbook. By the end you can design the zone topology for a hundred-endpoint estate, choose the resolver architecture, wire conditional forwarding to your datacenter, enforce the pattern fleet-wide so goodwill is never the control, and diagnose the handful of ways this silently breaks.

What problem this solves

Private Link is sold as “keep PaaS off the public internet.” That is the network half. The name half is where every production incident lives. Enable a private endpoint on a storage account and the account keeps its public FQDN and public IP; the endpoint adds a private IP reachable only from your network, but your application still connects to stappdata.blob.core.windows.net. If that name resolves to the public IP — which it does by default, everywhere, until you intervene — one of three things happens. If the firewall still allows public access, traffic takes the internet path and you paid for a private endpoint you are not using. If it is locked to “deny public,” the connection is refused or hangs. And in the cruelest variant, resolution works from the VNet where you tested but not from on-prem or an AKS pod, so it passes review and fails in production for a subset of clients.

What breaks without a deliberate DNS architecture: an application team creates a private endpoint, clicks past the DNS integration step (or points it at a zone not linked to their clients’ VNet), and ships. The NIC exists, the private IP is allocated, the resource firewall is locked down — and every client that was supposed to benefit now cannot resolve the name to the private IP. Weeks later it surfaces as “intermittent connectivity to storage” or “the on-prem batch job can’t reach the SQL database.” The record was never wrong; it was never visible to the client that needed it, because the zone containing it was not linked to that client’s resolver path. Multiply this by dozens of teams and hundreds of endpoints and you have a fleet-wide reliability tax that no amount of restarting fixes.

Who hits this: every enterprise adopting Private Link past a handful of endpoints — hardest on hybrid estates (on-prem needs an in-Azure forwarding target), multi-subscription landing zones (zone in the connectivity sub, endpoint in a spoke), container platforms (CoreDNS can silently bypass Azure DNS), and any org where application teams create endpoints. The fix is architectural: one canonical set of zones, automated registration via zone groups, a resolver for non-VNet clients, and policy that makes correct configuration the only configuration a team can ship.

The resolution problem restated per client class, because the answer differs by where the client sits:

Client location Uses which resolver by default Sees your Private DNS zones? What you must add
VM in a linked VNet (Azure-provided DNS) 168.63.129.16 (Azure DNS) Yes, if the zone is VNet-linked Just the VNet link (resolution-only)
VM in a VNet with a custom DNS server Your custom DNS No, unless the custom DNS forwards to 168.63.129.16 Forward all/PaaS suffixes to 168.63.129.16 (or Private Resolver inbound)
AKS pod (CoreDNS) CoreDNS → VNet DNS by default Yes, unless a forward override sends PaaS off-cluster Ensure CoreDNS forwards PaaS suffixes to Azure DNS, not an upstream
On-prem client over ExpressRoute/VPN On-prem DNS No — 168.63.129.16 is non-routable off its VNet Conditional forwarder → Private Resolver inbound endpoint
Client in a peered VNet in another tenant That tenant’s DNS No — cross-tenant zone links need RBAC you may not have Point that tenant’s DNS at your inbound endpoint, or link cross-tenant

Learning objectives

By the end of this article you can:

Prerequisites & where this fits

You should already understand Azure networking fundamentals: VNets and subnets, how VNet peering and hub-and-spoke connect them, and the difference between Azure-provided DNS and a custom DNS server on a VNet. You should know what a private endpoint is (a NIC with a private IP fronting a PaaS resource) and have created one. Familiarity with az in Cloud Shell, JSON output, and basic Bicep helps; the policy section assumes management groups and policy assignments. You need not be a DNS expert — we build the model from first principles — but be comfortable with zone, A record, CNAME, TTL, and conditional forwarder.

This sits at the intersection of the Networking and Governance tracks and is squarely an enterprise-scale topic. It assumes the endpoint mechanics from Azure Private Endpoint vs Service Endpoint: Secure PaaS Access and the Private Link-plus-DNS basics from Azure Private Link and Private DNS: Keeping PaaS Off the Public Internet. The single-zone mechanics — auto-registration, VNet links, split-horizon — are introduced in How to Set Up Azure Private DNS Zones: Auto-Registration, VNet Links and Split-Horizon; this article scales them to a fleet. It pairs with Hub-and-Spoke vs Virtual WAN: Choosing an Enterprise Cloud Network Topology (where the zones and resolver live), ExpressRoute Fundamentals: Circuits, Private vs Microsoft Peering and Connectivity Models (the path on-prem forwarding rides), and Azure Enterprise-Scale Landing Zone: Foundation for Large Organizations (the connectivity subscription and management-group structure the policy targets). The policy machinery leans on Azure Policy Effects Decoded: Deny vs Audit vs Modify vs DeployIfNotExists and Remediate Non-Compliant Resources: Remediation Tasks and Managed Identity Setup.

A quick map of who owns which layer, so you know whom to call when resolution breaks:

Layer What lives here Who usually owns it Failure classes it can cause
Private DNS zones + VNet links The canonical privatelink.* zones and their links Platform / connectivity team Client can’t see the zone; wrong zone linked
Private DNS zone group The endpoint→zone binding that writes the A record App team (endpoint) + platform (zone) No record; record in wrong zone; cross-sub RBAC gap
Private Resolver / forwarders Inbound/outbound endpoints, rulesets, forwarder VMs Platform / connectivity team On-prem/cross-tenant can’t resolve; ruleset gap
On-prem DNS Conditional forwarders for PaaS suffixes On-prem AD / network team Datacenter clients resolve public; one-direction gap
Custom VNet DNS / CoreDNS Whatever overrides Azure-provided DNS App team (AKS) / platform (VM DNS) PaaS suffix forwarded off-cluster to public
Azure Policy DINE definitions + initiative + assignment Governance / platform team Endpoints ship without zone groups; drift

Core concepts

Five mental models make every later decision obvious; internalize these and the rest is mechanics.

The name never changes; only the answer does. A private endpoint does not rename your resource; stappdata.blob.core.windows.net stays exactly that. What changes is the IP that name resolves to — but only for clients whose resolver path can see a Private DNS zone with the private A record. There is no “private FQDN”; you connect to the public name and arrange for it to resolve privately, and every design here controls which clients get the private answer.

Azure builds half the chain; you host the other half. Microsoft’s public resolvers return a CNAME from the public name to a privatelink.* alias; your responsibility is to host a Private DNS zone named exactly privatelink.<suffix> with the A record. The chain:

stappdata.blob.core.windows.net
  └─ CNAME stappdata.privatelink.blob.core.windows.net   (returned by Azure public DNS, always)
        └─ A  10.42.3.7   (resolves ONLY if the client can see privatelink.blob.core.windows.net with this record)

If the client can see a Private DNS zone named privatelink.blob.core.windows.net with that A record, it follows the CNAME to the private IP; if not, the chain falls through to the public A record. That is the whole mechanism, and every troubleshooting scenario later is a variation of one root cause: the client could not see the right zone, or the zone did not contain the right record.

The wire server makes it transparent — for VNet clients only. Azure’s platform DNS resolver, at the non-routable address 168.63.129.16, makes private resolution automatic for VMs using Azure-provided DNS: any VNet linked to a zone has that zone consulted automatically for its clients. You need no custom DNS on spokes for internal resolution — only a VNet link. But 168.63.129.16 is non-routable outside its VNet, so on-prem and cross-tenant clients need an in-Azure resolver (inbound endpoint or forwarder VM) to forward to.

Auto-registration belongs to the zone group, never your hand. A Private DNS zone group is a child of the private endpoint that binds it to zones and hands Azure the record lifecycle — write on creation, rewrite on IP change, delete on teardown. Hand-authored records rot on the first redeploy. Its --private-dns-zone parameter takes a full resource ID that can point at a zone in a different subscription, which is the mechanism of centralization.

Overrides are the enemy of consistency. Centralized zones only work if the client’s resolver path actually reaches 168.63.129.16 (directly or via a forwarder). Any layer that overrides DNS — a custom VNet DNS server, AKS CoreDNS with a forward block, an on-prem appliance with a stale forwarder — can send PaaS-suffix queries to a public resolver and bypass your zones. When resolution works on “some clients but not others,” an override is almost always the cause; auditing override points matters as much as configuring zones.

The vocabulary in one table

Before the deep sections, pin down every moving part — the mental model side by side:

Concept One-line definition Where it lives Why it matters at scale
Public FQDN The unchanged name your app/cert uses The resource Must resolve to the private IP for private clients
privatelink.* alias The CNAME target Azure returns Azure public DNS Bridges the public name to your hosted zone
Private DNS zone Zone you host with the private A record Connectivity sub (centralized) The thing the client must be able to see
VNet link (resolution) Attaches a zone to a VNet for lookups On the zone (registration-enabled false) How a spoke’s VMs see the zone
Zone group Endpoint child that writes the A record On the private endpoint Automates record lifecycle; cross-sub capable
168.63.129.16 Azure platform DNS (wire server) Every VNet, internally Auto-consults linked zones; non-routable off-VNet
Private Resolver inbound EP An IP on-prem/cross-tenant forwards to Hub, delegated subnet Lets non-VNet clients reach your zones
Private Resolver outbound EP Source for Azure→on-prem forwarding Hub, delegated subnet Sends internal-domain queries to on-prem
Forwarding ruleset Domain→target rules attached to a VNet Linked to spoke VNets Conditional forwarding without a VM
Conditional forwarder “For domain X, ask server Y” On-prem DNS or ruleset The hybrid glue in both directions
DINE policy DeployIfNotExists Azure Policy Landing-zone management group Auto-creates zone groups fleet-wide

The DNS chain, in detail

Everything starts with what resolves where, and the sequence depends on what the client’s resolver can see. Private path (correct): the resolver returns the privatelink CNAME, then — because it can see the linked privatelink.blob.core.windows.net zone with an A record — returns the private IP 10.42.3.7; traffic goes to the endpoint’s NIC over your network. Public path (the silent failure): the resolver returns the CNAME, cannot see any privatelink.blob zone, follows the CNAME to Azure’s public A record, and returns a public IP; traffic tries the internet and, if the firewall denies public, hangs or is refused.

The table below is the single most important reference in this article: the per-service zone-name catalogue — the sub-resource (group-id) you specify on the endpoint, and the exact Private DNS zone name that must host the record. Getting the zone name wrong is the number-one rollout bug: the zone group writes the record into a zone nobody queries, and resolution silently falls through to public.

Service Sub-resource (group-id) Private DNS zone name
Blob (Storage) blob privatelink.blob.core.windows.net
File (Storage) file privatelink.file.core.windows.net
Table (Storage) table privatelink.table.core.windows.net
Queue (Storage) queue privatelink.queue.core.windows.net
Data Lake Gen2 (Storage) dfs privatelink.dfs.core.windows.net
Static website (Storage) web privatelink.web.core.windows.net
Key Vault vault privatelink.vaultcore.azure.net
Azure SQL Database sqlServer privatelink.database.windows.net
Azure SQL Managed Instance managedInstance privatelink.<region>.database.windows.net (per-region)
Cosmos DB (Core/SQL API) Sql privatelink.documents.azure.com
Cosmos DB (MongoDB API) MongoDB privatelink.mongo.cosmos.azure.com
Cosmos DB (Cassandra API) Cassandra privatelink.cassandra.cosmos.azure.com
Cosmos DB (Table API) Table privatelink.table.cosmos.azure.com
App Service / Functions sites privatelink.azurewebsites.net
Azure Container Registry registry privatelink.azurecr.io (+ regional data endpoint)
Service Bus / Event Hubs / Relay namespace privatelink.servicebus.windows.net
Event Grid (topic) topic privatelink.eventgrid.azure.net
Azure Cache for Redis redisCache privatelink.redis.cache.windows.net
PostgreSQL / MySQL Flexible Server postgresqlServer / mysqlServer privatelink.postgres.database.azure.com / privatelink.mysql.database.azure.com
Azure Kubernetes Service (API server) management privatelink.<region>.azmk8s.io (per-region)
Azure Monitor (AMPLS) azuremonitor a set: privatelink.monitor.azure.com, .oms.opinsights.azure.com, .ods.opinsights.azure.com, .agentsvc.azure-automation.net, privatelink.blob.core.windows.net
Azure AI Search searchService privatelink.search.windows.net
Azure OpenAI / Cognitive Services account privatelink.cognitiveservices.azure.com (+ openai.azure.com)

Three traps on this table catch experienced teams:

Trap The mistake The reality
Key Vault suffix Assuming the zone is privatelink.vault.azure.net (the public name) It is privatelink.**vaultcore**.azure.net — different word
Storage is not one endpoint One endpoint/zone for a storage account Blob, file, table, queue, dfs, web are independent sub-resources, each its own FQDN, endpoint, and zone
Regional zones Using a global zone for SQL MI / AKS SQL Managed Instance and AKS embed the region in the zone name; a global zone never matches

Storage sub-resources are separate zones. A storage account is not one private endpoint: static websites add a web zone you never provisioned, Data Lake Gen2 adds dfs, and an app using blob and file needs both endpoints and both zones. Cosmos DB creates an A record per partition region — a three-region account produces multiple records the zone group manages. AMPLS needs a set of zones, not one. Sovereign/Government clouds use different suffixes (*.core.usgovcloudapi.net). When in doubt, Microsoft’s “Azure Private Endpoint DNS configuration” reference is the source of truth — do not guess a suffix.

Centralized vs decentralized: the blast-radius decision

You have two topologies for where the privatelink zones live. Understand the trade before you pour concrete — migrating later means re-pointing hundreds of zone groups.

Dimension Decentralized (zone per spoke) Centralized (zone in hub)
Zone copies One privatelink.blob... per spoke Exactly one, in the connectivity subscription
Record drift High — N places to diverge Single source of truth
Cross-spoke resolution Awkward; needs peering + links anyway Native via VNet links
RBAC blast radius Each team self-serves their zone Platform team owns zones; spokes own endpoints
Failure if a zone is misconfigured One spoke affected Potentially the fleet
Adding a spoke Recreate every zone in the new spoke One entry in a Terraform map + apply
On-prem forwarding Ambiguous (which spoke’s zone?) One resolver, one set of zones to forward to
Policy enforcement N zone IDs to parameterize One canonical zone ID per service
Operational owner Distributed, inconsistent Centralized, consistent

Centralized wins for any estate past a handful of spokes, and it is the model in the Azure enterprise-scale landing zone. The blast-radius concern is real — a fat-fingered hub record affects everyone — but you mitigate it with policy-as-code and zone groups (so humans never write records), not by accepting forty divergent copies. Decentralized’s only genuine advantage is isolation, and that isolation is precisely what makes cross-spoke and on-prem resolution painful. The model this article builds: the connectivity/hub subscription owns the canonical zones; each spoke owns its endpoints; Azure Policy stitches them together; the zone group bridges the subscription boundary.

The rare cases where decentralized still makes sense:

If your situation is… Then choose… Because…
Enterprise landing zone, many spokes, hybrid Centralized Single source of truth; on-prem forwards to one place
A few unrelated VNets, no hub, no on-prem Either (lean centralized) Overhead is low; centralized still future-proofs
Strict per-tenant/per-BU isolation mandated Decentralized per boundary Blast radius must not cross the boundary
Multiple Azure tenants you don’t control jointly Zone per tenant + cross-tenant forwarding RBAC can’t span the tenant boundary cleanly
Regulated workload needing independent change control Decentralized for that workload Change to shared zones is a shared-fate risk

Building the centralized zones and VNet links

Create one copy of each privatelink zone in a dedicated DNS resource group in the connectivity subscription, then link every spoke VNet to each zone.

HUB_RG="rg-connectivity-dns"
LOCATION="eastus2"

# One canonical zone per PaaS suffix you use (Private DNS zones are global; location is ignored)
for zone in \
  privatelink.blob.core.windows.net \
  privatelink.file.core.windows.net \
  privatelink.vaultcore.azure.net \
  privatelink.database.windows.net \
  privatelink.azurewebsites.net ; do
  az network private-dns zone create \
    --resource-group "$HUB_RG" \
    --name "$zone"
done

Now link the spokes. A VNet link with registration-enabled false is a resolution-only link — what you want for a shared zone. Auto-registration (true) is for VM hostnames in one VNet and has no business in a privatelink zone; enabling it there pollutes the zone with VM records.

az network private-dns link vnet create \
  --resource-group "$HUB_RG" \
  --zone-name privatelink.blob.core.windows.net \
  --name link-spoke-payments \
  --virtual-network "$SPOKE_PAYMENTS_VNET_ID" \
  --registration-enabled false

The registration flag is a frequent point of confusion:

registration-enabled What it does Use it for Never use it for
false (resolution-only) VNet’s clients can look up records in the zone privatelink.* shared zones (the norm)
true (auto-registration) VMs in that VNet auto-register their hostnames as A records A private zone for VM hostnames in one VNet privatelink.* zones — pollutes and confuses

A hard platform limit shapes the design: a VNet links to at most 1,000 Private DNS zones for resolution (and registers to at most one). The ~25 common suffixes never approach that, but it is why you host one zone per suffix linked to many VNets — not one zone per endpoint. Each zone holds 25,000 record sets, far more than any realistic endpoint count per suffix.

At fleet scale this is a Terraform for_each over the cross-product of zones and spokes. One block creates the zones, one creates every link:

locals {
  privatelink_zones = [
    "privatelink.blob.core.windows.net",
    "privatelink.file.core.windows.net",
    "privatelink.vaultcore.azure.net",
    "privatelink.database.windows.net",
    "privatelink.azurewebsites.net",
  ]
}

resource "azurerm_private_dns_zone" "this" {
  for_each            = toset(local.privatelink_zones)
  name                = each.value
  resource_group_name = azurerm_resource_group.dns.name
}

# Cartesian product: every canonical zone linked to every spoke VNet
resource "azurerm_private_dns_zone_virtual_network_link" "this" {
  for_each = {
    for pair in setproduct(local.privatelink_zones, keys(var.spoke_vnets)) :
    "${pair[0]}::${pair[1]}" => { zone = pair[0], vnet = pair[1] }
  }

  name                  = "link-${each.value.vnet}"
  resource_group_name   = azurerm_resource_group.dns.name
  private_dns_zone_name = azurerm_private_dns_zone.this[each.value.zone].name
  virtual_network_id    = var.spoke_vnets[each.value.vnet]
  registration_enabled  = false
}

Adding a new spoke is now one entry in var.spoke_vnets and a terraform apply — every zone links automatically. That is the operational payoff of centralization: onboarding a spoke drops from “recreate N zones” to “add one line.”

The same in Bicep — a zone and a resolution-only link:

param spokeVnetId string
param linkName string = 'link-spoke-payments'

resource blobZone 'Microsoft.Network/privateDnsZones@2020-06-01' = {
  name: 'privatelink.blob.core.windows.net'
  location: 'global'          // Private DNS zones are always global
}

resource blobLink 'Microsoft.Network/privateDnsZones/virtualNetworkLinks@2020-06-01' = {
  parent: blobZone
  name: linkName
  location: 'global'
  properties: {
    registrationEnabled: false
    virtualNetwork: { id: spokeVnetId }
  }
}

Automating A-record registration with the zone group

Never write the A record yourself. A Private DNS zone group binds an endpoint to zones and hands Azure the record lifecycle — what makes “hundreds of endpoints” tractable: every endpoint self-registers, self-updates, and self-cleans, with records in the one canonical zone regardless of the endpoint’s subscription.

# 1. Create the endpoint in the spoke (storage blob example)
az network private-endpoint create \
  --name pe-stappdata-blob \
  --resource-group rg-app-payments \
  --vnet-name vnet-spoke-payments \
  --subnet snet-privateendpoints \
  --private-connection-resource-id "$STORAGE_ID" \
  --group-id blob \
  --connection-name conn-stappdata-blob

# 2. Bind it to the CENTRAL zone (note: --private-dns-zone is a FULL resource ID)
az network private-endpoint dns-zone-group create \
  --resource-group rg-app-payments \
  --endpoint-name pe-stappdata-blob \
  --name default \
  --private-dns-zone "$HUB_ZONE_ID_BLOB" \
  --zone-name privatelink-blob

The critical detail: --private-dns-zone takes a full resource ID that can point at a zone in a different subscription — the mechanism of centralization: the spoke owns the endpoint, the connectivity subscription owns the zone, the zone group bridges them. (--zone-name is just a local label, not the zone FQDN.) The zone group can list multiple privateDnsZoneConfigs, so a single AMPLS endpoint binds all five Monitor zones at once.

The full lifecycle the zone group manages, versus hand-writing records:

Event With a zone group (automated) With hand-written A records
Endpoint created A record written automatically You must remember to add it
Private IP changes (recreate/move) Record rewritten automatically Stale record → resolves to a dead IP
Endpoint deleted Record deleted automatically Orphan record lingers, mis-resolves
Multi-region (Cosmos) partitions All partition records managed You track N records by hand
Cross-subscription zone Native via full resource ID Manual, error-prone, drifts
Audit / compliance Config is declarative on the endpoint No single source of truth

In Bicep, endpoint and zone group are declared together so the binding is atomic and reviewable in a PR:

param location string
param subnetId string
param storageId string
param blobZoneId string   // full resource ID of the hub's privatelink.blob zone

resource pe 'Microsoft.Network/privateEndpoints@2023-11-01' = {
  name: 'pe-stappdata-blob'
  location: location
  properties: {
    subnet: { id: subnetId }
    privateLinkServiceConnections: [ {
      name: 'conn-stappdata-blob'
      properties: {
        privateLinkServiceId: storageId
        groupIds: [ 'blob' ]
      }
    } ]
  }
}

resource zoneGroup 'Microsoft.Network/privateEndpoints/privateDnsZoneGroups@2023-11-01' = {
  parent: pe
  name: 'default'
  properties: {
    privateDnsZoneConfigs: [ {
      name: 'privatelink-blob'
      properties: { privateDnsZoneId: blobZoneId }   // points at the HUB zone, any subscription
    } ]
  }
}

The resolver choice: Private Resolver vs custom forwarders vs Private DNS Zones

Three mechanisms can answer a private query, and the right one depends on who is asking. For an in-VNet VM using Azure-provided DNS, a directly-linked Private DNS Zone is the whole answer — link the zone, and 168.63.129.16 resolves transparently, no resolver and no cost beyond the zones. But it only serves clients using Azure-provided DNS whose VNet is linked; on-prem cannot forward to 168.63.129.16 (non-routable), and a custom-DNS VNet bypasses it. It is necessary but not sufficient for hybrid.

For on-prem and cross-tenant clients you need something they can forward to, and the choice is Azure DNS Private Resolver versus custom DNS forwarder VMs. The Private Resolver is a managed, zone-redundant service (no VMs to patch) offering an inbound endpoint (a private IP on-prem forwards to, which resolves against your linked zones), an outbound endpoint (source for Azure→on-prem forwarding), and forwarding rulesets (domain→target rules attached to VNets, replacing per-VM conditional forwarders). Both endpoints require dedicated /28 subnets delegated to Microsoft.Network/dnsResolvers, one endpoint per subnet. Custom DNS forwarder VMs are the legacy alternative — a pair of DNS VMs forwarding to 168.63.129.16 for Azure and to on-prem for internal names — which you must patch, scale, monitor, and make HA yourself. For pure Private Link resolution the Private Resolver is strictly better operationally; forwarder VMs survive only for DNS features the resolver lacks (complex views, non-Azure integrations, DNS-based filtering).

The full comparison:

Dimension Directly-linked Private DNS Zone Azure DNS Private Resolver Custom DNS forwarder VMs
Serves in-VNet Azure-DNS clients Yes (the whole answer) Yes (via link to the resolver’s VNet) Yes (VMs forward to 168.63.129.16)
Serves on-prem clients No (168.63.129.16 non-routable) Yes — on-prem forwards to inbound EP Yes — on-prem forwards to the VMs
Serves cross-tenant clients No (needs cross-tenant link RBAC) Yes — point their DNS at inbound EP Yes — point their DNS at the VMs
Azure→on-prem forwarding No Yes — outbound EP + ruleset Yes — VM conditional forwarders
Operational burden Minimal Minimal (managed, zone-redundant) High (patch, scale, monitor, HA)
High availability Platform-managed Built-in, zone-redundant You build it (2+ VMs)
Cost Zones + queries Per endpoint/hour + per million queries VM compute + licensing + ops
Subnet requirement None /28 delegated subnet per endpoint Normal subnet, 2 VMs
When to choose VNet-only, no hybrid Hybrid or cross-tenant (default) Legacy/feature-parity edge cases

A decision table:

If you need… Choose Notes
VNet clients only, no on-prem, no custom DNS Directly-linked zones Nothing else required
On-prem clients to resolve private endpoints Private Resolver inbound endpoint On-prem conditional-forwards to it
Azure VMs to resolve on-prem internal names Private Resolver outbound endpoint + ruleset Ruleset rule: corp.internal → on-prem DNS
Both directions of hybrid Both endpoints + ruleset The standard hub deployment
Cross-tenant resolution you can’t RBAC-link Point the other tenant’s DNS at your inbound EP Resolution happens in the zone-owning tenant
DNS features beyond forwarding (filtering, complex views) Custom forwarder VMs (or resolver + NVA) Rare; weigh the ops cost

The query path each client takes, so you can trace any failure to the layer that owns it:

Client Query path (correct) Which mechanism makes it work
VM, Azure-provided DNS Client → 168.63.129.16 → linked zone → private IP Directly-linked Private DNS Zone
VM, custom VNet DNS Client → custom DNS → 168.63.129.16 → linked zone Custom DNS forwards to 168.63.129.16
AKS pod Pod → CoreDNS → VNet DNS (168.63.129.16) → zone CoreDNS must not override PaaS suffixes
On-prem client Client → on-prem DNS → inbound EP → 168.63.129.16 → zone Private Resolver inbound endpoint
Cross-tenant client Client → their DNS → your inbound EP → zone Inbound endpoint (resolution stays in your tenant)
Azure VM needing on-prem name Client → 168.63.129.16 → outbound EP/ruleset → on-prem DNS Private Resolver outbound endpoint + ruleset

Deploying the Private Resolver

The resolver lives in the hub. Reserve the /28 subnets up front — retrofitting subnet space into a running hub is painful.

HUB_VNET="vnet-hub"
HUB_RG="rg-connectivity"
LOC="eastus2"

# Dedicated, delegated subnets (one per endpoint), minimum /28
az network vnet subnet create -g "$HUB_RG" --vnet-name "$HUB_VNET" \
  --name snet-dns-inbound --address-prefixes 10.10.0.0/28 \
  --delegations Microsoft.Network/dnsResolvers
az network vnet subnet create -g "$HUB_RG" --vnet-name "$HUB_VNET" \
  --name snet-dns-outbound --address-prefixes 10.10.0.16/28 \
  --delegations Microsoft.Network/dnsResolvers

# The resolver itself
az dns-resolver create -g "$HUB_RG" -n dnspr-hub -l "$LOC" \
  --id-virtual-network "$(az network vnet show -g $HUB_RG -n $HUB_VNET --query id -o tsv)"

# Inbound endpoint (the IP on-prem forwards TO)
az dns-resolver inbound-endpoint create -g "$HUB_RG" --dns-resolver-name dnspr-hub \
  -n inbound -l "$LOC" \
  --ip-configurations '[{"privateIpAllocationMethod":"Dynamic","subnet":{"id":"'"$(az network vnet subnet show -g $HUB_RG --vnet-name $HUB_VNET -n snet-dns-inbound --query id -o tsv)"'"}}]'

# Outbound endpoint (source for Azure -> on-prem forwarding)
az dns-resolver outbound-endpoint create -g "$HUB_RG" --dns-resolver-name dnspr-hub \
  -n outbound -l "$LOC" \
  --id "$(az network vnet subnet show -g $HUB_RG --vnet-name $HUB_VNET -n snet-dns-outbound --query id -o tsv)"

A forwarding ruleset attached to the outbound endpoint sends internal-domain queries to on-prem; link it to the VNets whose clients need on-prem resolution:

# Ruleset + a rule: forward corp.internal to the on-prem DNS servers
az dns-resolver forwarding-ruleset create -g "$HUB_RG" -n frs-hub -l "$LOC" \
  --outbound-endpoints '[{"id":"'"$OUTBOUND_EP_ID"'"}]'

az dns-resolver forwarding-rule create -g "$HUB_RG" --ruleset-name frs-hub \
  -n rule-corp-internal --domain-name "corp.internal." \
  --forwarding-rule-state Enabled \
  --target-dns-servers '[{"ipAddress":"10.50.0.10","port":53},{"ipAddress":"10.50.0.11","port":53}]'

# Link the ruleset to a spoke VNet so its clients use these rules
az dns-resolver vnet-link create -g "$HUB_RG" --ruleset-name frs-hub \
  -n link-spoke-payments --id "$SPOKE_PAYMENTS_VNET_ID"

The resolver’s own deployment constraints drive hub subnet and IP planning:

Private Resolver constraint Value Design implication
Subnet per endpoint One endpoint per subnet, delegated to Microsoft.Network/dnsResolvers Two /28 subnets (inbound + outbound) minimum
Minimum subnet size /28 (11 usable) Reserve the space in the hub IP plan up front
Endpoints per resolver Multiple inbound/outbound supported One resolver per region usually suffices
Resolver per VNet One DNS Private Resolver per VNet Deploy it in the hub VNet
Availability Zone-redundant, platform-managed No HA to build (unlike forwarder VMs)
Region scope Regional resource Add a second resolver only for another region’s hybrid

The ruleset replaces configuring conditional forwarders on every custom DNS VM. Key ruleset facts and limits:

Ruleset property Value / behavior Why it matters
Rules per ruleset Up to 1,000 Ample for internal domains
Rulesets linked per VNet Multiple can be linked Compose rules from several rulesets
Domain match Longest-suffix match wins More specific rule overrides a broader one
A . (root) rule Forwards everything to a target Use to send all unknown queries somewhere specific
Rule state Enabled / Disabled Toggle a rule without deleting it
Fallthrough Non-matching queries → default Azure resolution Private zones still resolve normally

On-prem and cross-tenant resolution over ExpressRoute

VNet clients resolve transparently through 168.63.129.16; on-prem and cross-tenant clients cannot (that address is non-routable outside its VNet) and need an in-Azure resolver to forward to — the Private Resolver’s inbound endpoint. The direction teams forget is on-prem into Azure: on on-prem DNS, create a conditional forwarder for the public PaaS suffix (blob.core.windows.net, not privatelink.blob...) pointing at the inbound endpoint IP.

$inbound = "10.10.0.4"   # Private Resolver inbound endpoint IP in the hub

"blob.core.windows.net",
"file.core.windows.net",
"vaultcore.azure.net",
"database.windows.net" | ForEach-Object {
  Add-DnsServerConditionalForwarderZone `
    -Name $_ `
    -MasterServers $inbound `
    -ReplicationScope "Forest"
}

The mechanism: on-prem asks for stappdata.blob.core.windows.net and forwards it to the inbound endpoint; inside Azure, 168.63.129.16 returns the public CNAME, follows it into your linked privatelink zone, and returns the private IP. On-prem never references the privatelink name — it only forwards the public suffix — and traffic rides your existing ExpressRoute private peering or VPN, since the resolver IP is a normal hub private address. The crucial subtlety is which name on-prem forwards — the public suffix, never the privatelink alias:

On-prem conditional forwarder for… Result Verdict
blob.core.windows.net → inbound EP Azure returns CNAME→privatelink→private A. Correct. ✅ Do this
privatelink.blob.core.windows.net → inbound EP Azure resolves the alias directly; works but fragile, and misses the public CNAME semantics ⚠️ Avoid — forward the public suffix
blob.core.windows.net → public Azure DNS On-prem gets the public IP; private endpoint bypassed ❌ The classic bug
No forwarder at all On-prem resolves public everywhere ❌ Datacenter never uses private

The same pattern solves cross-tenant resolution: if a peered VNet in another tenant cannot be RBAC-linked to your zones across the boundary, point that tenant’s resolver at your inbound endpoint as a conditional forwarder for the PaaS suffixes — resolution still happens in the zone-owning tenant. For the global multi-region case, see Cross-Region Private Link and DNS for Global Active-Active Applications.

Bidirectional hybrid forwarding is two independent configurations — miss one and half of hybrid works:

Direction Configure on… Points at… For which names
On-prem → Azure (resolve private endpoints) On-prem DNS Private Resolver inbound EP Public PaaS suffixes (blob.core.windows.net, …)
Azure → on-prem (resolve internal names) Private Resolver ruleset / custom DNS On-prem DNS servers Internal domains (corp.internal, …)

Enforcing the pattern fleet-wide with Azure Policy

Manual zone groups do not survive contact with a dozen teams — the first endpoint created without one is a silent public-resolution bug that surfaces weeks later as a connectivity ticket. Close the gap with Azure Policy’s DeployIfNotExists (DINE) effect: the policy watches for new private endpoints and creates the zone group pointing at your canonical zone, no human in the loop. For the full effect taxonomy, see Azure Policy Effects Decoded: Deny vs Audit vs Modify vs DeployIfNotExists.

Microsoft ships built-in DINE definitions (search the catalog for “Configure private endpoints … to use private DNS zones”) — one per service, bundled into an initiative assigned at the landing-zone management group, each parameterized with the canonical zone’s resource ID. That bundling is the decision in Policy Definitions vs Initiatives: When to Bundle Controls into a Set.

The rule shape, so you know what the engine evaluates and deploys:

{
  "if": {
    "allOf": [
      { "field": "type", "equals": "Microsoft.Network/privateEndpoints" },
      {
        "count": {
          "field": "Microsoft.Network/privateEndpoints/privateLinkServiceConnections[*].groupIds[*]",
          "where": {
            "field": "Microsoft.Network/privateEndpoints/privateLinkServiceConnections[*].groupIds[*]",
            "equals": "blob"
          }
        },
        "greaterOrEquals": 1
      }
    ]
  },
  "then": {
    "effect": "deployIfNotExists",
    "details": {
      "type": "Microsoft.Network/privateEndpoints/privateDnsZoneGroups",
      "roleDefinitionIds": [
        "/providers/Microsoft.Authorization/roleDefinitions/4d97b98b-1d4f-4787-a291-c67834d212e7"
      ],
      "deployment": {
        "properties": {
          "mode": "incremental",
          "parameters": {
            "privateDnsZoneId": { "value": "[parameters('privateDnsZoneId')]" },
            "privateEndpointName": { "value": "[field('name')]" },
            "location": { "value": "[field('location')]" }
          },
          "template": { "...": "ARM template that creates the privateDnsZoneGroups child resource" }
        }
      }
    }
  }
}

Two things make or break this.

The managed identity needs roles across two subscriptions. The roleDefinitionIds above is Network Contributor (4d97b98b-…), needed to write the zone group on the endpoint. Because the canonical zone lives in another subscription, the identity also needs Private DNS Zone Contributor (b12aa53e-…) scoped to the hub DNS resource group — and when you assign at a management group, that hub grant is not implied. This cross-subscription grant is the single most common reason a correctly-written DINE policy silently fails to register records.

Role Role definition ID Scope needed Why
Network Contributor 4d97b98b-1d4f-4787-a291-c67834d212e7 The endpoint’s subscription/MG Write the privateDnsZoneGroups child on the endpoint
Private DNS Zone Contributor b12aa53e-6015-4669-85d0-8515ebb3ae7f The hub DNS resource group Write the A record into the cross-sub canonical zone

DINE only fires on new and updated resources. Endpoints predating the assignment are flagged non-compliant but not fixed until you run a remediation task — mechanics in Remediate Non-Compliant Resources: Remediation Tasks and Managed Identity Setup.

# Find the assignment, then remediate existing non-compliant endpoints
ASSIGNMENT_ID=$(az policy assignment show \
  --name "deploy-pe-privatedns" \
  --scope "/providers/Microsoft.Management/managementGroups/mg-landingzones" \
  --query id -o tsv)

az policy remediation create \
  --name "remediate-pe-privatedns-blob" \
  --policy-assignment "$ASSIGNMENT_ID" \
  --definition-reference-id "configurePrivateEndpointBlob" \
  --resource-discovery-mode ReEvaluateCompliance

Once assigned and remediated, the outcome is structural: every endpoint created under that management group, by any team, lands its record in the one canonical zone. Governance, not goodwill.

Layer a Deny policy alongside DINE for defense in depth — deny an endpoint that manually specifies a different (non-canonical) zone. The effects compose:

Effect What it guarantees Gap it leaves Pair with
DeployIfNotExists Every endpoint gets a zone group to the canonical zone Doesn’t stop a manual different-zone binding A Deny on non-canonical zones
Deny (non-canonical zone) No endpoint binds a wrong zone Doesn’t create the binding if absent DINE to create the correct one
Audit Visibility into non-compliant endpoints Fixes nothing DINE + a compliance dashboard

Architecture at a glance

Picture the connectivity subscription as the center of gravity. A dedicated DNS resource group holds exactly one Private DNS zone per PaaS suffix — privatelink.blob.core.windows.net, privatelink.vaultcore.azure.net, privatelink.database.windows.net, and so on — each global with no location. Every spoke VNet across every application subscription attaches to each zone by a resolution-only link (registration-enabled false). That fan-out of links lets any VM in any spoke, using Azure-provided DNS at 168.63.129.16, resolve a private endpoint’s public FQDN to its private IP with no per-spoke zone copies.

Now overlay the endpoints. In each spoke, teams create private endpoints in a dedicated snet-privateendpoints subnet, each carrying a zone group whose privateDnsZoneId is a full resource ID pointing across the subscription boundary at the matching canonical zone. Azure writes the A record on creation, rewrites it on IP change, and removes it on deletion — no human touches a record. Above all of this, an Azure Policy initiative at the landing-zone management group creates any missing zone group automatically, its identity holding Network Contributor on the spokes and Private DNS Zone Contributor on the hub DNS group.

Finally, the hybrid edge. In the hub VNet, an Azure DNS Private Resolver occupies two delegated /28 subnets: an inbound endpoint (a private IP on-prem DNS conditional-forwards the public PaaS suffixes to, over ExpressRoute) and an outbound endpoint with a ruleset sending internal-domain queries back to on-prem. Trace a request: an on-prem batch job asks for stappdata.blob.core.windows.net; on-prem forwards blob.core.windows.net to the inbound endpoint over ExpressRoute; inside Azure, 168.63.129.16 returns the public CNAME, follows it into the hub’s linked privatelink.blob zone, and returns 10.42.3.7; the job connects over the same circuit. Every arrow — spoke link, cross-sub zone group, policy, inbound forwarder — exists so one public name resolves privately for one more class of client. The system is a fan-out of links and a single set of zones, its correctness enforced by policy rather than trusted to discipline.

Real-world scenario

Meridian Freight, a logistics company, ran Azure for two years with private endpoints created ad hoc by nine application teams. Every team had, at some point, created a privatelink.blob.core.windows.net zone in their own spoke subscription — nine copies, each linked only to its own VNet. It mostly worked, because each team resolved its own storage from its own VNet. The cracks showed when the platform team stood up a shared ExpressRoute circuit and the on-prem data-warehouse team needed to pull from three different teams’ storage accounts over the private path.

The symptom was maddening: the on-prem ETL job resolved one team’s storage to a private IP and the other two to public IPs, and the public ones hung because those accounts had “deny public access” set. On-prem DNS had a single conditional forwarder for blob.core.windows.net pointing at a custom DNS VM in team A’s spoke — so only team A’s zone (linked to team A’s VNet, which the VM could see) resolved privately; teams B and C had identical zones the forwarder never reached. Three engineers spent a day convinced it was firewall or routing before someone ran nslookup from on-prem and saw the public A records for B and C.

The platform team took the incident as the mandate to centralize. Over two weeks they created one canonical set of five privatelink zones in a new rg-connectivity-dns; linked all nine spoke VNets to each via a Terraform setproduct (44 links from one apply); deployed an Azure DNS Private Resolver in the hub with an inbound endpoint at 10.10.0.4; and re-pointed the on-prem forwarders for all PaaS suffixes at that single inbound IP over ExpressRoute. They then assigned the Microsoft DINE initiative at the landing-zone management group, parameterized with the five canonical zone IDs, granted its identity Private DNS Zone Contributor on the hub DNS group, and ran a remediation task that re-pointed all 130-odd existing endpoints and backfilled records for the two dozen with none.

The cutover was not instant — the one surprise. Forwarders and clients had cached the old public answers for the record TTL, and a legacy appliance held a 3600-second negative cache, so for the first hour some on-prem clients still resolved public. Once caches expired, nslookup from on-prem returned private IPs for all three teams’ storage and the ETL job completed over the private path for the first time. Three months later a tenth team onboarded: one line in the Terraform spoke map, one apply, and their new endpoints self-registered with zero DNS tickets. The lesson in the runbook: “Nine copies of a zone is nine ways to be inconsistent. One zone, one resolver, policy on top — and ‘the app can’t reach storage’ stops being a recurring incident.”

The migration as a table, because the order and the caching gotcha are the lesson:

Phase Action Effect Watch-out
0 Nine spoke-local zones, ad hoc endpoints On-prem resolves only team A privately Inconsistent by construction
1 Create 5 canonical zones in hub Source of truth exists Zones empty until linked/bound
2 Terraform setproduct: 44 VNet links Every spoke can see every zone Resolution-only links (registration false)
3 Deploy Private Resolver, inbound EP On-prem has one place to forward to /28 subnet delegated up front
4 Re-point on-prem forwarders → inbound EP On-prem forwards public suffixes to Azure Forward the public suffix, not privatelink
5 Assign DINE initiative + grant hub RBAC New endpoints self-register Cross-sub Private DNS Zone Contributor grant
6 Remediation task 130 existing endpoints backfilled DINE doesn’t touch pre-existing without this
7 Cutover On-prem resolves all teams privately TTL/negative-cache delay ~1 hour

Advantages and disadvantages

The centralized-zone-plus-policy model has a clear shape of benefits and costs. Weigh it honestly before committing an estate.

Advantages Disadvantages
One source of truth for every privatelink record — no drift across spokes A mistake in a hub zone (or its RBAC) can affect the whole fleet
Onboarding a spoke is one Terraform line + apply Requires disciplined IaC and a platform team that owns the zones
On-prem forwards to one resolver for all PaaS suffixes The Private Resolver and its /28 subnets must be planned into the hub
Zone groups automate record lifecycle — no hand-written records to rot Cross-subscription RBAC for the DINE identity is easy to get wrong
Azure Policy makes correct config the only shippable config DINE has evaluation latency; pre-existing endpoints need remediation
Cross-tenant resolution solvable via inbound-endpoint forwarding Cross-tenant RBAC for direct zone links is genuinely awkward
Consistent security posture — no spoke silently resolving public Cutover and record changes are subject to DNS TTL/caching delays
Scales to thousands of endpoints (25k records/zone, 1k zones/VNet) Concentrates operational and change-control risk in the connectivity sub

The model is right for enterprise landing zones with many spokes, a hybrid footprint, and teams that create their own endpoints. It is over-engineered for a handful of unrelated VNets with no on-prem and no governance requirement, where directly-linked zones per VNet are simpler. The disadvantages are all manageable — IaC discipline, planned subnet space, correct RBAC, TTL awareness — but only if you know they exist, which is the point of enumerating them.

Hands-on lab

Reproduce the centralized pattern at minimum scale: one canonical zone in a “hub” resource group, a storage account with a private endpoint in a “spoke,” a cross-resource-group zone group that writes the record, and end-to-end verification from a VM. Run in Cloud Shell (Bash); delete the resource groups at the end. (Two resource groups in one subscription keep the lab cheap; the cross-subscription mechanics are identical — only the zone’s resource ID changes.)

Step 1 — Variables and resource groups.

SUB=$(az account show --query id -o tsv)
LOC=eastus2
HUB_RG=rg-lab-dns
SPOKE_RG=rg-lab-spoke
VNET=vnet-lab-spoke
SA=stlab$RANDOM        # globally-unique storage account name
az group create -n $HUB_RG -l $LOC -o table
az group create -n $SPOKE_RG -l $LOC -o table

Step 2 — Spoke VNet with a private-endpoint subnet.

az network vnet create -g $SPOKE_RG -n $VNET -l $LOC \
  --address-prefixes 10.60.0.0/16 \
  --subnet-name snet-pe --subnet-prefixes 10.60.1.0/24 -o table

Expected: a VNet with one subnet snet-pe.

Step 3 — The canonical blob zone in the “hub,” linked to the spoke (resolution-only).

az network private-dns zone create -g $HUB_RG -n privatelink.blob.core.windows.net -o table

az network private-dns link vnet create -g $HUB_RG \
  --zone-name privatelink.blob.core.windows.net \
  --name link-spoke --virtual-network \
  "$(az network vnet show -g $SPOKE_RG -n $VNET --query id -o tsv)" \
  --registration-enabled false -o table

Expected: the zone, then a link with registrationEnabled: false.

Step 4 — A storage account with public access disabled (so only private resolves).

az storage account create -g $SPOKE_RG -n $SA -l $LOC \
  --sku Standard_LRS --kind StorageV2 --public-network-access Disabled -o table
STORAGE_ID=$(az storage account show -g $SPOKE_RG -n $SA --query id -o tsv)

Step 5 — The private endpoint in the spoke.

az network private-endpoint create -g $SPOKE_RG -n pe-$SA-blob \
  --vnet-name $VNET --subnet snet-pe \
  --private-connection-resource-id "$STORAGE_ID" \
  --group-id blob --connection-name conn-$SA-blob -o table

Expected: an endpoint with a private IP in 10.60.1.0/24.

Step 6 — The zone group binding the endpoint to the HUB zone (writes the A record).

HUB_ZONE_ID=$(az network private-dns zone show -g $HUB_RG \
  -n privatelink.blob.core.windows.net --query id -o tsv)

az network private-endpoint dns-zone-group create -g $SPOKE_RG \
  --endpoint-name pe-$SA-blob --name default \
  --private-dns-zone "$HUB_ZONE_ID" --zone-name privatelink-blob -o table

Step 7 — Validate the record landed in the hub zone, automatically.

az network private-dns record-set a list -g $HUB_RG \
  --zone-name privatelink.blob.core.windows.net \
  --query "[].{name:name, ip:aRecords[0].ipv4Address}" -o table

Expected: one row, name = <SA>, ip = 10.60.1.x — the A record you never wrote, created by the zone group. That is the whole point of the pattern in one command.

Step 8 — (Optional) Prove resolution from a VM in the spoke. Create a tiny VM in the VNet and resolve the name:

az vm create -g $SPOKE_RG -n vm-lab --image Ubuntu2204 --size Standard_B1s \
  --vnet-name $VNET --subnet snet-pe --admin-username azureuser \
  --generate-ssh-keys --public-ip-address "" -o table
# From the VM (via 'az vm run-command' or SSH through a bastion):
az vm run-command invoke -g $SPOKE_RG -n vm-lab --command-id RunShellScript \
  --scripts "nslookup $SA.blob.core.windows.net"

Expected: the output shows the privatelink CNAME and a 10.60.1.x private A record — never a public IP.

Validation checklist. You created one canonical zone, linked it resolution-only to a spoke, created an endpoint in that spoke, bound it to the hub zone across resource groups via a zone group, and watched Azure write the A record automatically — then resolved the public FQDN to the private IP from a VM. The lab steps mapped to the production concepts:

Step What you did What it proves Production analogue
3 Zone in hub RG, resolution-only link Zones live centrally; spokes only link Canonical zones in the connectivity sub
4 --public-network-access Disabled Only the private path can work Locked-down PaaS firewall
6 Zone group → hub zone ID Cross-boundary binding writes the record Cross-subscription zone group
7 Record appears with no manual add Zone-group auto-registration No hand-written A records at scale
8 nslookup returns private IP The chain resolves end to end The “did it actually work?” check

Cleanup (avoid lingering charges).

az group delete -n $SPOKE_RG --yes --no-wait
az group delete -n $HUB_RG --yes --no-wait

Cost note. A Standard_LRS account and one private endpoint are a few rupees for the hour; the optional B1s VM is a few rupees more. Private DNS zones cost a token amount per zone per month plus a tiny per-query charge. Deleting both resource groups stops everything. The whole lab is well under ₹50 for an hour.

Common mistakes & troubleshooting

This is the playbook — the part you bookmark. First the scannable table, then the confirm-command detail for the entries that bite hardest. Every row reduces to the same root cause: the client could not see the right zone, or the zone did not contain the right record.

# Symptom Root cause Confirm (exact cmd / portal path) Fix
1 nslookup from a spoke VM returns a public IP for a PaaS FQDN The zone isn’t linked to that VNet (or the wrong zone name) az network private-dns link vnet list -g <hub> --zone-name <zone>; check the VNet is present Create a resolution-only link for that VNet
2 Record never appears in the zone after creating the endpoint No zone group, or zone group points at the wrong zone az network private-endpoint dns-zone-group show ... Create/fix the zone group with the correct zone ID
3 DINE policy shows the endpoint compliant but no record exists Managed identity lacks Private DNS Zone Contributor on the hub Deployment failed in Activity Log; check role assignments on hub DNS RG Grant b12aa53e-… on the hub DNS scope; re-remediate
4 Resolves privately from VMs but public from on-prem On-prem has no (or wrong) conditional forwarder to the inbound EP Resolve-DnsName on-prem shows public; check on-prem forwarders Add forwarder: public suffix → inbound endpoint IP
5 Resolves privately from VMs but public from AKS pods only CoreDNS forward block sends PaaS suffix off-cluster kubectl get configmap coredns-custom -n kube-system -o yaml Scope the forward to internal domains only
6 Record written into a zone but clients still get public Wrong zone name — e.g. vault.azure.net instead of vaultcore Compare the zone name to the per-service table Recreate the zone with the exact suffix; rebind
7 Custom-DNS VNet resolves everything public Custom DNS server doesn’t forward to 168.63.129.16 Check the custom DNS server’s forwarders Forward PaaS suffixes (or .) to 168.63.129.16 / inbound EP
8 On-prem resolves internal names fine but Azure can’t reach on-prem No outbound endpoint / ruleset for the internal domain az dns-resolver forwarding-rule list --ruleset-name <frs> Add a ruleset rule: internal domain → on-prem DNS
9 After a cutover, some clients still resolve public for ~an hour DNS TTL / negative caching of the old public answer Check record TTL; a resolver may hold a negative cache Wait out the TTL; lower TTL before planned cutovers
10 Storage resolves for blob but not file (or web/dfs) Only the blob sub-resource has an endpoint/zone List the account’s private endpoints and their group-ids Create the file/web/dfs endpoint + its zone + zone group
11 Cosmos/SQL-MI/AKS resolves public despite a zone Wrong (global) zone for a per-API / per-region service Verify the zone name includes the region/API Use privatelink.<region>.database.windows.net etc.
12 Azure Monitor Private Link half-works (some telemetry public) Only one of the AMPLS zones provisioned List the five AMPLS zones; check each is linked Provision and link the full AMPLS zone set
13 Endpoint deleted but the old A record still resolves Record was hand-written, not managed by a zone group az network private-dns record-set a list shows an orphan Delete the orphan; always use zone groups going forward
14 New spoke’s VMs resolve public though the zone exists Spoke VNet never linked to the canonical zones az network private-dns link vnet list per zone Add the VNet to var.spoke_vnets; terraform apply

The expanded form, with the full reasoning for the entries that cost the most time:

1. nslookup from a spoke VM returns a public IP. The VNet is not linked to the privatelink zone (or is linked to a wrong-named one); Azure DNS only consults zones linked to that VNet. Confirm: az network private-dns link vnet list -g rg-connectivity-dns --zone-name privatelink.blob.core.windows.net — is the spoke’s VNet present? Fix: create a resolution-only link — in the centralized model, a missing entry in the Terraform setproduct; add the spoke and apply.

3. DINE compliant but no record. The identity wrote (or tried to write) the zone group but lacked Private DNS Zone Contributor on the cross-subscription hub DNS group, so the record write failed while the endpoint shows the zone group. Confirm: the DNS resource group’s Activity Log shows a failed deployment; az role assignment list --scope <hub DNS RG id> --assignee <principalId> shows no Private DNS Zone Contributor. Fix: grant b12aa53e-… on the hub scope and re-run remediation.

4. Private from VMs, public from on-prem. On-prem DNS has no forwarder for the public PaaS suffix (or points it at public Azure DNS). Confirm: on-prem, Resolve-DnsName stappdata.blob.core.windows.net returns a public IP; inspect the on-prem forwarders. Fix: add a forwarder for blob.core.windows.net (the public suffix) → the inbound endpoint IP, over ExpressRoute/VPN; repeat per suffix.

5. Private from VMs, public from AKS pods. CoreDNS has a custom forward block sending queries (often .) to an on-prem or public upstream, so pods bypass 168.63.129.16. Confirm: kubectl get configmap coredns-custom -n kube-system -o yaml shows a broad forward . <upstream>. Fix: scope the forward to only the internal domain and let the rest fall through to the VNet’s Azure DNS — detailed in Troubleshooting DNS on AKS: CoreDNS, Custom Forwarders, Private DNS Zones, and NXDOMAIN Storms.

6. Record in a zone, clients still public. The zone name is wrong — the record sits in privatelink.vault.azure.net while clients follow the CNAME to privatelink.vaultcore.azure.net, which doesn’t exist for them. Confirm: compare the zone name character-for-character against the per-service table. Fix: create the zone with the exact suffix, rebind the zone group, delete the wrong zone.

9. Post-cutover public for an hour. Clients and forwarders cached the old public answer for the record TTL, and some resolvers hold a negative cache. Confirm: check the A-record TTL and intermediate resolver caching; the delay matches the largest TTL in the path. Fix: there is no instant flip — lower record/zone TTLs before a planned migration.

When resolution is correct but the request is still refused, cross-reference Fixing Azure Storage 403 Errors: Firewalls, Private Endpoints, RBAC & SAS and Key Vault 403 Forbidden: Untangling Firewall, RBAC, Purge Protection, and Soft-Delete Recovery. When you cannot tell whether it is DNS or routing, Diagnosing Connectivity with Network Watcher: Connection Monitor, Connection Troubleshoot and Next Hop localizes it.

Best practices

Security notes

The security-relevant controls and what each protects:

Control Mechanism Protects against
Least-privilege policy identity Scoped Network + Private DNS Zone Contributor Over-broad automation access
Restricted zone write access + PIM RBAC on the connectivity sub Estate-wide FQDN redirection
PaaS public access disabled --public-network-access Disabled Traffic taking the public path despite PE
Inbound-endpoint exposure control NSG + circuit scoping Untrusted networks querying private zones
Cross-tenant forwarding (not linking) Conditional forwarder → inbound EP Granting excess cross-tenant RBAC
Resolution-only VNet links registration-enabled false Rogue VM records polluting the zone
Compliance scan for zone groups Policy Audit + daily report Silent public-resolving endpoints

Cost & sizing

Private Link DNS at scale is cheap relative to what it protects, but the line items are worth knowing so nothing surprises you. Private DNS zones cost a small flat amount per zone plus a per-million-query charge — with ~25 zones this is a rounding error, tens to low hundreds of rupees a month for the estate. Azure DNS Private Resolver is billed per endpoint/hour plus per-million-queries; two endpoints run continuously, and it is materially cheaper than a pair of HA DNS VMs with their compute, licensing, and ops. Private endpoints cost per endpoint/hour plus per-GB — the dominant Private Link line item at scale, but a function of your architecture, not your DNS design. Custom forwarder VMs (if you go legacy) cost two VMs plus licensing plus the engineering time to patch and make them HA. Azure Policy has no engine charge; the zone groups DINE creates are free.

Sizing, not fake numbers: one Private Resolver in the hub serves the whole region’s hybrid resolution — you do not scale it per spoke; add a second only for another region’s hybrid. The /28 per endpoint gives 11 usable addresses, far more than needed; the real constraint is hub address-space planning, not query capacity. Zones hold 25,000 record sets and a VNet links 1,000 zones, so realistic estates never approach those ceilings.

Cost driver What you pay for Rough monthly shape Scales with
Private DNS zones Per zone + per million queries Tens–low hundreds of ₹ total Number of suffixes (fixed ~25) + query volume
Private Resolver Per endpoint/hour + per million queries Modest per resolver Regions (not spokes)
Private endpoints Per endpoint/hour + per-GB Dominant at scale (hundreds of PEs) Number of endpoints
Custom forwarder VMs (legacy) 2× VM compute + licensing + ops Higher than Private Resolver HA requirement
Azure Policy DINE Engine free; created zone groups free ~₹0 Nothing (fixed)

Interview & exam questions

1. A private endpoint exists and the resource firewall denies public access, but the app can’t connect. What’s the first thing you check? DNS resolution: the app is almost certainly resolving the public FQDN to the public IP because it cannot see a linked privatelink zone with the private A record. Run nslookup from the client — a public IP confirms it. The endpoint and firewall are red herrings until DNS resolves privately.

2. Why does adding a private endpoint not change the name your application connects to? A private endpoint projects a NIC with a private IP but does not rename the resource; the SDK, connection string, and TLS certificate all keep the public FQDN. Private Link DNS is entirely about making that unchanged public name resolve to the private IP for the clients that need it.

3. What is a Private DNS zone group and why should you never hand-write A records? A zone group is a child resource of the private endpoint that binds it to one or more zones and lets Azure manage the A-record lifecycle — writing it on creation, rewriting on IP change, deleting on teardown. Hand-written records go stale on the first redeploy or IP change and orphan on deletion; the zone group is the only reliable mechanism at scale, and it can bind a zone in another subscription via full resource ID.

4. Compare centralized and decentralized Private DNS zone topologies. Centralized hosts one canonical zone per suffix in the connectivity subscription, linked to all spokes — one source of truth and native cross-spoke/on-prem resolution, but a hub mistake can affect the fleet. Decentralized puts a copy in each spoke — isolated blast radius, but drift-prone and awkward for cross-spoke/on-prem. Centralized wins past a handful of spokes; mitigate blast radius with policy and zone groups, not divergent copies.

5. When do you choose Azure DNS Private Resolver over directly-linked Private DNS Zones? Directly-linked zones serve in-VNet Azure-DNS clients and need nothing else. Add a Private Resolver when non-VNet clients must resolve private endpoints — on-prem (which can’t reach 168.63.129.16) or cross-tenant — via its inbound endpoint, and/or when Azure must resolve on-prem names via its outbound endpoint and ruleset. It is a managed, zone-redundant alternative to custom DNS VMs.

6. How does on-prem resolve a private endpoint’s FQDN, and which name does it forward? On-prem DNS has a conditional forwarder for the public PaaS suffix (blob.core.windows.net, not privatelink.blob...) pointing at the Private Resolver inbound endpoint, reachable over ExpressRoute/VPN. Azure’s 168.63.129.16 returns the public CNAME, follows it into the linked privatelink zone, and returns the private IP. On-prem never references the privatelink alias.

7. What two RBAC grants does a DINE policy for private-endpoint DNS need, and why? Network Contributor on the endpoint’s scope (to write the privateDnsZoneGroups child) and Private DNS Zone Contributor on the hub DNS resource group (to write the record into the cross-subscription zone). The second is not implied when you assign at a management group and is the top reason a correct-looking DINE policy fails to register records.

8. A DINE policy shows a private endpoint compliant, yet no A record exists. Cause? The policy attempted the zone group but its managed identity lacked Private DNS Zone Contributor on the hub DNS scope, so the record write failed while the zone-group config shows on the endpoint. Check the DNS resource group’s Activity Log and the identity’s role assignments; grant the role and re-remediate.

9. Name three counterintuitive Private DNS zone names. Key Vault is privatelink.vaultcore.azure.net, not vault.azure.net (the public name). SQL Managed Instance and AKS embed the region (privatelink.<region>.database.windows.net / .azmk8s.io). AMPLS needs a set of zones (monitor, oms, ods, agentsvc, blob), not one. A wrong suffix lands the record in a zone nobody queries and resolution falls through to public.

10. Why does resolution work from VMs but not AKS pods in the same VNet? CoreDNS has a custom forward block sending PaaS-suffix queries to a public or on-prem upstream, so pods bypass 168.63.129.16 and never consult the linked zone, while node VMs using Azure-provided DNS resolve fine. Scope the CoreDNS forward to only the internal domain.

11. After migrating records to a centralized zone, why do some clients still resolve public for a while? DNS caching — clients and forwarders hold the old public answer for the record TTL, and some resolvers keep a negative cache. There is no instant flip; plan for the TTL window and lower TTLs before a planned cutover.

12. How do you make correct DNS configuration the only one a team can ship? Assign a DeployIfNotExists initiative at the landing-zone management group that auto-creates the zone group to the canonical zone, pair it with a Deny on non-canonical zones, grant the identity the two roles, and run a remediation task for existing endpoints. Correct configuration becomes structural — governance, not goodwill.

These map to AZ-700 (Network Engineer)design and implement private access to Azure services — as the primary cert, AZ-305 (Solutions Architect) for the topology and landing-zone design, and AZ-104 for the Private DNS and endpoint fundamentals. A compact cert map:

Question theme Primary cert Objective area
Private Link DNS chain, zone groups AZ-700 Private access to Azure services
Private Resolver, inbound/outbound, rulesets AZ-700 Design/implement hybrid DNS
Centralized vs decentralized topology AZ-305 Design network architecture
On-prem forwarding over ExpressRoute AZ-700 Hybrid connectivity + DNS
DINE policy + cross-sub RBAC AZ-305 / AZ-104 Governance; policy; RBAC
Per-service zone names, storage sub-resources AZ-104 Private DNS + private endpoints

Quick check

  1. A private endpoint is created and the storage firewall denies public access, but the app times out. What is the single most likely cause, and how do you confirm it in one command?
  2. Which name does an on-prem DNS server conditionally forward to the Private Resolver inbound endpoint — the public suffix or the privatelink alias — and why?
  3. True or false: you should enable registration-enabled true when linking a spoke VNet to a shared privatelink.blob.core.windows.net zone.
  4. A DINE policy reports a private endpoint as compliant, but the A record never appears in the hub zone. Name the most likely missing piece.
  5. Resolution works from a spoke VM but returns a public IP from AKS pods in the same VNet. What is going on?

Answers

  1. DNS is resolving the public FQDN to the public IP because the client cannot see a linked privatelink zone with the private A record. Confirm with nslookup <resource>.<suffix> from the client — a public IP proves it. The endpoint and firewall are fine; the zone isn’t linked (or the record/zone name is wrong).
  2. The public suffix (e.g. blob.core.windows.net), never the privatelink alias. Forwarding the public suffix lets Azure’s 168.63.129.16 return the public CNAME and follow it into your linked privatelink zone; on-prem never needs to know the privatelink name, and this preserves the correct CNAME semantics.
  3. False. Use registration-enabled false (resolution-only). Auto-registration is for VM hostnames in a single VNet; on a privatelink zone it pollutes the zone with VM records and is a misconfiguration.
  4. Private DNS Zone Contributor on the hub DNS resource group for the policy’s managed identity. The identity had Network Contributor to write the zone group on the endpoint but lacked rights to write the record into the cross-subscription canonical zone, so the deployment failed while the endpoint shows compliant. Grant the role and re-remediate.
  5. AKS CoreDNS has a custom forward block sending PaaS-suffix queries to a public or on-prem upstream, so pods bypass 168.63.129.16 and never consult the linked private zone, while the VM using Azure-provided DNS resolves correctly. Scope the CoreDNS forward to only the internal domain that needs it.

Glossary

Next steps

You can now design, automate, and enforce Private Link DNS for an enterprise estate. Build outward.

AzurePrivate LinkPrivate EndpointDNSPrivate ResolverPrivate DNS ZonesGovernanceAzure Policy
Need this built for real?

Vinod is a Senior Cloud Architect (22+ yrs) — available for Azure / AWS / GCP architecture, landing zones, and migrations.

Work with me

Comments

Keep Reading