Terraform on Azure at Scale: Reusable Modules, Multi-Environment Landing Zones & SRE Practices

There is a moment on every serious Azure programme where a single main.tf stops being infrastructure and becomes a liability. It happened for one client the week their auditor asked a simple question — “show me that dev and prod are built from the same code, and prove prod can’t be changed by accident” — and the answer was a shrug, three folders of copy-pasted resources that had quietly drifted apart, and one state file on a build agent nobody wanted to touch. Nothing was wrong with the HCL. What was wrong was that there was no platform: no reusable module library, no per-environment isolation, no guardrails, no observability that came for free with every resource. This lesson is how you build that platform on Azure, and it is the capstone that ties the whole course together.

You already know the parts. You can write a resource, a variable, a module; you know what remote state and locking are; you have met Terragrunt. Here we assemble them into an enterprise landing zone: a versioned modules/ library (network, compute, aks, data) that many teams consume, per-environment root configurations (dev, uat, prod) that compose those modules and pass outputs between them, each environment with its own remote-state key in an Azure Storage Account, promotion that walks a change dev → uat → prod, management-group Azure Policy guardrails stamped over subscriptions used as blast-radius boundaries, and SRE baked into the code — standardised tags and names, a diagnostic setting to Log Analytics on every resource, metric alerts, cost budgets, least-privilege RBAC, and drift detection. Every pattern is shown twice where it matters (raw tfvars-per-env and Terragrunt), with real azurerm HCL you can paste and run.

This is deliberately opinionated because a platform is a set of decisions, not a pile of options. Read the prose once for the reasoning; then the tables — the repo layout, the modules-vs-Terragrunt matrix, the ALZ/AVM decision grid, the SRE-controls catalogue and the troubleshooting map — are the reference you keep open while you build. By the end you will be able to stand up a landing zone that an auditor, an SRE and a new hire can all read, and answer that “prove it” question with a git log and a terraform plan.

What you’ll build

The running example is Contoso, a company standing up its first governed Azure estate in australiaeast. We build a platform with three moving planes. The module plane is a modules/ directory holding four reusable, versioned building blocks: network (a VNet with subnets and NSGs), compute (VM scale sets or App Service), aks (a managed cluster), and data (a hardened Storage Account / database). Each module has typed inputs, validated variables, sensitive outputs where needed, and — crucially — emits its own diagnostic setting so observability is a property of the module, not an afterthought. The environment plane is a set of root configurations, one per environment (dev, uat, prod), each of which composes the modules, passing one module’s outputs into the next, and each of which owns a separate remote-state key so a mistake in dev is physically incapable of touching prod. The governance plane sits above: management groups carrying Azure Policy assignments that every subscription below inherits, with each environment ideally living in its own subscription as a hard boundary.

Why Terraform rather than the portal, az CLI, or ARM/Bicep? Because a landing zone’s whole value is that it is reproducible, reviewable and enforced. The portal produces snowflakes nobody can recreate; a pile of az commands is a script with no notion of desired state or drift; Bicep is excellent Azure-native IaC but stops at the resource-manager boundary — Terraform’s provider ecosystem lets the same tool and the same state model govern Azure AD/Entra, Azure DevOps or GitHub, Datadog, and the Azure resources together, which is exactly what a platform team needs. And Terraform’s module + remote-state model is the cleanest expression of the two things a landing zone must have: reuse (one module, many environments) and isolation (one state per environment).

Here is the leap this lesson is about — the difference between a config and a platform, stated as symptoms you can recognise on your own estate:

By the end you can build every row of the right-hand column with real HCL.

Learning objectives

By the end of this lesson you will be able to:

• Lay out a platform repository — a modules/ library and per-environment root configurations — and justify what lives where and why each layer gets its own state.
• Compose modules in a root config, passing outputs from one module (e.g. a Log Analytics workspace ID) into others, and reason about the implicit ordering that creates.
• Isolate environments two ways — the tfvars-per-env pattern and Terragrunt (dependency, per-env state keys, DRY generation) — and choose between them deliberately.
• Design per-environment remote state on an Azure Storage Account (one container, a key per environment) and drive a promotion flow dev → uat → prod.
• Apply Azure landing-zone concepts — management groups, subscriptions-as-boundaries, and Azure Policy guardrails via azurerm_management_group_policy_assignment — and decide when to adopt the Azure Verified Modules and CAF enterprise-scale ALZ modules rather than roll your own.
• Bake SRE into code — standardised tagging/naming, diagnostic settings to Log Analytics on every resource, azurerm_monitor_metric_alert, budgets, least-privilege azurerm_role_assignment, drift detection and state hygiene.
• Test modules with fmt, validate, tflint, checkov and native terraform test, and wire them into a CI gate.

Prerequisites & where this fits

This is the Azure capstone of the Terraform Zero-to-Hero course. It assumes the platform-building primitives are already familiar and pulls them together into one real estate. You will get the most from it having already authored a reusable module with typed inputs, outputs and versioning, worked through Terragrunt fundamentals — DRY, remote state and dependencies, seen the multi-environment 3-tier centerpiece with approval gates and the DRY multi-account/environment pattern, and understood remote state at scale. Where those lessons teach a technique in isolation, this one shows all of them driven by Azure landing-zone requirements at once.

A note on versions: everything targets Terraform ≥ 1.6 (the 1.9/1.10 line current in 2026) with the azurerm provider ~> 4.0 — note that azurerm 4.x made subscription_id mandatory on the provider block, and azurerm_monitor_diagnostic_setting uses the enabled_log block, both reflected below. OpenTofu is a drop-in for the Terraform CLI throughout; the module and state model are identical. Terragrunt references assume its current direction (units/stacks). Assume you have az login working against a tenant where you can create resource groups, and — for the governance layer — rights at a management group scope.

Because this lesson pins azurerm ~> 4.0, the handful of 4.x changes that bite an upgrader are worth having in front of you — every one of them is reflected in the HCL below:

Here is the map of companion lessons and what each carries, so you know where to go deep on any one plane:

From a single config to a platform: the repo that scales

The repository is the architecture. Before a line of HCL, the folder layout decides your reuse story (is there one module library or three copies?), your isolation story (how many state files, and what’s the blast radius of each?), and your governance story (where do the management-group and policy definitions live, and who can apply them?). Get the tree right and the rest of the lesson is filling it in.

Here is the canonical layout for the Contoso platform. It separates the modules library (reused, versioned, never applied directly) from per-environment roots (applied, each with its own state) and from a platform/management layer (management groups + policy, a different blast radius and often a different team):

contoso-landing-zone/
├── modules/                      # the library — reusable, versioned, never applied directly
│   ├── network/                  # vnet, subnets, nsgs (+ its own diagnostic setting)
│   │   ├── main.tf
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   └── README.md
│   ├── compute/                  # vmss / app service
│   ├── aks/                      # managed cluster
│   └── data/                     # storage account / database (hardened)
│
├── environments/                 # per-env ROOT configs — each has its own remote state
│   ├── dev/
│   │   ├── versions.tf           # required_providers + backend "azurerm" {}
│   │   ├── locals.tf             # naming + common_tags
│   │   ├── main.tf               # composes modules/*  ← the root module
│   │   ├── variables.tf
│   │   ├── outputs.tf
│   │   ├── dev.auto.tfvars       # env-specific values (SKUs, counts, CIDRs)
│   │   └── backend.hcl           # partial backend config: key = "dev.terraform.tfstate"
│   ├── uat/                      # identical structure, uat.auto.tfvars, key = uat...
│   └── prod/                     # identical structure, prod.auto.tfvars, key = prod...
│
├── platform/                     # the governance plane — separate state, separate blast radius
│   └── management-groups/        # mgmt groups + azurerm_management_group_policy_assignment
│       ├── main.tf
│       ├── variables.tf
│       └── backend.hcl           # key = "platform-mg.terraform.tfstate"
│
├── tests/                        # native terraform test (*.tftest.hcl)
├── .tflint.hcl
└── .checkov.yaml

Read the tree by state boundaries, because that is what actually matters at 2 a.m. Every leaf directory under environments/ and platform/ is a root module with its own state key — that is the unit of plan/apply and the unit of blast radius. The modules/ directory has no state of its own; it is pure library code that roots reference by source. The one rule that saves you: never terraform apply inside modules/ — modules are consumed, not run.

Two structural decisions deserve emphasis. First, the platform/management layer is its own root with its own state, deliberately separated from the environments. Management-group policy assignments have tenant-wide reach; you do not want a routine dev apply to be able to touch them, and you usually want a different, smaller group of humans able to apply them. That is blast-radius thinking made concrete in the folder tree. Second, environments are siblings, not a shared config with a workspace switch. Directory-per-environment (rather than CLI workspaces) is the right default for a landing zone because each env is an independently-reviewable folder, can differ structurally when it must, and its state key is explicit in a backend.hcl you can read — no terraform.workspace interpolation to reason about under pressure.

The diagram traces the whole platform left to right: the modules/ library on the left is consumed by three per-environment roots, each of which writes its own state key (with a blob lease as the lock) and applies into its own subscription under management-group policy, while the SRE plane — diagnostics to Log Analytics, alerts, and least-privilege RBAC — is stamped across everything. The six badges are the six load-bearing decisions of the lesson.

The modules library: network, compute, aks, data

A landing-zone module is not “a wrapper around one resource”. It is a coherent, named unit of infrastructure with a contract: typed inputs, validated where it matters, sensitive outputs where secrets flow, a README, and — the Contoso house rule — a diagnostic setting it creates for itself so that consuming the module automatically means the resource is observable. The library is versioned (git tags, or published to a private registry) so an environment pins network v1.4.0 and upgrades on its own schedule; dev can trial v1.5.0 while prod stays on v1.4.0, and because they share the module’s shape, the upgrade is a reviewable diff, not a rewrite.

Here is the network module — small enough to read, real enough to run. Note the for_each over a subnets map (so one module makes any number of subnets), the validation on address_space, and the self-contained diagnostic setting keyed off a log_analytics_workspace_id input:

# modules/network/variables.tf
variable "name_prefix"         { type = string }
variable "location"            { type = string }
variable "resource_group_name" { type = string }

variable "address_space" {
  type = list(string)
  validation {
    condition     = length(var.address_space) > 0
    error_message = "Provide at least one CIDR for the VNet address space."
  }
}

variable "subnets" {
  type = map(object({
    address_prefixes = list(string)
  }))
}

variable "log_analytics_workspace_id" {
  type        = string
  description = "LAW to send this VNet's diagnostics to — makes observability a module property."
}

variable "tags" {
  type    = map(string)
  default = {}
}

# modules/network/main.tf
resource "azurerm_virtual_network" "this" {
  name                = "vnet-${var.name_prefix}"
  location            = var.location
  resource_group_name = var.resource_group_name
  address_space       = var.address_space
  tags                = var.tags
}

resource "azurerm_subnet" "this" {
  for_each             = var.subnets
  name                 = "snet-${each.key}"
  resource_group_name  = var.resource_group_name
  virtual_network_name = azurerm_virtual_network.this.name
  address_prefixes     = each.value.address_prefixes
}

# Observability is a property of the module, not a later chore.
resource "azurerm_monitor_diagnostic_setting" "vnet" {
  name                       = "diag-vnet-to-law"
  target_resource_id         = azurerm_virtual_network.this.id
  log_analytics_workspace_id = var.log_analytics_workspace_id

  enabled_log { category_group = "allLogs" }
  metric      { category = "AllMetrics" }
}

# modules/network/outputs.tf
output "vnet_id" {
  value = azurerm_virtual_network.this.id
}

output "subnet_ids" {
  description = "Map of subnet name → resource ID, for consumers (compute, aks)."
  value       = { for k, s in azurerm_subnet.this : k => s.id }
}

That is the shape every module in the library follows. The catalogue below is the library contract at a glance — what each module wraps, its key inputs, and the outputs downstream modules consume:

The load-bearing idea is composition by output passing. A root doesn’t just instantiate modules side by side; it wires them — the monitoring module’s log_analytics_workspace_id output feeds every other module’s diagnostic input, and the network module’s subnet_ids feed compute and aks. That wiring is also the dependency graph: Terraform sees that network references monitoring’s output and orders them automatically — no depends_on needed. This table is the wiring map you keep in your head:

How a root references a module — its source and version pin — is the other half of “versioned library”. The four forms, and when each is right:

The rule that makes upgrades safe: always pin (a ?ref tag or a version constraint), and let dev adopt a new module version before uat and prod — the same promotion flow you use for infrastructure applies to the library itself.

For the deep mechanics of module contracts — semver, optional() object attributes, sensitive propagation, publishing to a registry — see the authoring reusable modules lesson; here the point is how the library is consumed and wired into a landing zone.

Multi-environment: tfvars-per-env vs Terragrunt

You now have a library and a single root that composes it. The multi-environment problem is: how do you get dev, uat and prod from that root without duplicating the composition three times and without their states ever colliding? There are two mainstream answers, and choosing between them is the central decision of this section.

Answer 1 — tfvars-per-env (native Terraform). Keep a full root config per environment folder (they are near-identical), and drive the differences with a per-env *.auto.tfvars and a per-env partial backend config. The backend block is left empty in code and completed at init time with a -backend-config file, so each environment gets the same container but a different key:

# environments/dev/versions.tf
terraform {
  required_version = ">= 1.6.0"
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
  backend "azurerm" {}          # ← empty: completed by -backend-config at init
}

provider "azurerm" {
  features {}
  subscription_id = var.subscription_id   # required in azurerm 4.x
}

# environments/dev/backend.hcl  →  terraform init -backend-config=backend.hcl
resource_group_name  = "rg-tfstate-platform"
storage_account_name = "sttfstateplatform001"
container_name       = "tfstate"
key                  = "dev.terraform.tfstate"   # uat/prod differ ONLY here
use_azuread_auth     = true                        # AAD auth, no storage keys

# environments/dev/dev.auto.tfvars
subscription_id          = "00000000-dev-0000-0000-000000000000"
location                 = "australiaeast"
vnet_address_space       = ["10.10.0.0/16"]
subnets                  = { app = { address_prefixes = ["10.10.1.0/24"] }, aks = { address_prefixes = ["10.10.16.0/20"] } }
storage_replication_type = "LRS"      # prod would be "GZRS"
log_retention_days       = 30         # prod would be 90
monthly_budget           = 500        # prod would be larger

The strength is that it is just Terraform — no extra tool, every environment folder is self-contained and reviewable, and juniors already understand it. The weakness is repetition: the versions.tf, provider block and main.tf composition are copy-pasted across dev/, uat/ and prod/, so a change to the composition is a three-place edit, and it is easy for the folders to drift.

Answer 2 — Terragrunt (DRY). Terragrunt keeps the composition in one place and generates the backend and provider per environment, so the only thing that lives in each env folder is its inputs. A root terragrunt.hcl defines the backend once (with a per-env key derived from the path) and generates the provider; each child terragrunt.hcl includes it, points at a module, declares its dependency on other units, and supplies inputs:

# live/terragrunt.hcl  (root — defined ONCE)
remote_state {
  backend = "azurerm"
  generate = { path = "backend.tf", if_exists = "overwrite_terragrunt" }
  config = {
    resource_group_name  = "rg-tfstate-platform"
    storage_account_name = "sttfstateplatform001"
    container_name       = "tfstate"
    key                  = "${path_relative_to_include()}/terraform.tfstate"  # per-unit key
    use_azuread_auth     = true
  }
}

generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "azurerm" {
  features {}
  subscription_id = "${local.env.locals.subscription_id}"
}
EOF
}

locals {
  env = read_terragrunt_config(find_in_parent_folders("env.hcl"))
}

# live/dev/network/terragrunt.hcl  (a unit)
include "root" { path = find_in_parent_folders() }

terraform { source = "${get_repo_root()}/modules//network" }

dependency "monitoring" {
  config_path = "../monitoring"
  mock_outputs = {                         # lets `plan` run before monitoring exists
    log_analytics_workspace_id = "/subscriptions/0/resourceGroups/m/providers/Microsoft.OperationalInsights/workspaces/m"
  }
}

inputs = {
  name_prefix                = "contoso-platform-dev"
  address_space              = ["10.10.0.0/16"]
  subnets                    = { app = { address_prefixes = ["10.10.1.0/24"] } }
  log_analytics_workspace_id = dependency.monitoring.outputs.log_analytics_workspace_id
}

Now dev/uat/prod differ only in their env.hcl and unit inputs; the backend, provider and module source are DRY. The cost is the extra tool, the wrapper indirection when debugging, and a dependency/mock_outputs model you must understand. The Terragrunt mechanics — generate, dependency, run-all, mocks — are covered in depth in Terragrunt fundamentals and applied with approval gates in the multi-environment 3-tier lesson.

Choose with this matrix, not by fashion:

The decision rule Contoso used: start native while there are three environments and one platform team; adopt Terragrunt when the number of (environment × stack) combinations makes the copy-paste unmanageable or when you split into per-team subscriptions and need generated per-account backends — the situation the DRY multi-account lesson is written for.

Either way, remote state is per-environment. This is the isolation that makes the platform safe, and it is worth stating as a table because getting the key scheme right is what stops a dev apply from planning to destroy prod:

The azurerm backend takes a blob lease on the specific blob during an operation, so locking is per-key: dev and prod can apply concurrently because they lease different blobs. (For real isolation many teams go further and put each environment’s state in a separate subscription’s Storage Account, so even the backend is a blast boundary — but one account with per-env keys is the pragmatic default.)

Promotion — dev → uat → prod. The point of shared modules and per-env tfvars is that a change is promoted, not rewritten. You prove it in dev, then run the same module code against uat with uat’s tfvars, then prod behind an approval. Nothing about the resources is hand-edited between environments; only the tfvars (SKU, counts, CIDRs, replication) change.

The mechanics of the gates — plan/apply separation, storing the plan artifact, requiring an environment approval — belong to your CI system (Azure Pipelines environments or GitHub Actions environments with required reviewers). That CI/CD wiring for Terraform on Azure DevOps pipelines is a lesson in its own right; conceptually it is: plan in CI, publish the plan, require an approval on prod, apply the approved plan (never re-plan at apply time).

Azure landing zones: management groups, subscriptions & policy

Everything so far governs resources. A landing zone also governs the containers those resources live in — the management-group hierarchy and subscriptions — and stamps Azure Policy as guardrails so that non-compliant resources are denied or flagged regardless of who applies what. This is the governance plane, and in the Contoso repo it lives in platform/management-groups/ as its own root with its own state, because its blast radius is the whole tenant.

The mental model: management groups are a tree above subscriptions; a policy or RBAC assignment at a management group is inherited by every subscription beneath it. Subscriptions are the hard boundary — for billing, for quota, and (the reason we care) for blast radius: putting dev, uat and prod in separate subscriptions means a runaway apply, a compromised credential, or a policy mistake is contained to one environment. The landing-zone pattern is a small management-group hierarchy (a platform group for shared services, a landing-zones group for workloads) with policy assigned high and subscriptions slotted underneath.

The Contoso hierarchy follows the CAF enterprise-scale archetype — a small, legible tree where policy lives high and workloads slot in per environment:

Here is a real management-group policy assignment. It uses azurerm_management_group_policy_assignment to bind two built-in Azure Policies at a management-group scope — “Allowed locations” (deny anything outside the approved regions) and a required-tag policy — looked up by name via a data source so you never hardcode a policy GUID wrong:

# platform/management-groups/main.tf
data "azurerm_policy_definition" "allowed_locations" {
  display_name = "Allowed locations"
}

resource "azurerm_management_group_policy_assignment" "allowed_locations" {
  name                 = "allowed-locations"
  management_group_id  = var.platform_mg_id
  policy_definition_id = data.azurerm_policy_definition.allowed_locations.id
  description          = "Deny resource creation outside approved Australian regions."

  parameters = jsonencode({
    listOfAllowedLocations = {
      value = ["australiaeast", "australiasoutheast"]
    }
  })
}

data "azurerm_policy_definition" "require_tag" {
  display_name = "Require a tag on resources"
}

resource "azurerm_management_group_policy_assignment" "require_cost_center" {
  name                 = "require-cost-center-tag"
  management_group_id  = var.platform_mg_id
  policy_definition_id = data.azurerm_policy_definition.require_tag.id
  parameters           = jsonencode({ tagName = { value = "cost_center" } })
}

Assign these once, high in the tree, and every subscription below inherits them: a deny policy makes a non-compliant terraform apply fail at apply time with a clear policy reason, which is exactly the guardrail you want — the platform enforces the rule, not a code reviewer’s memory. Where to put what:

When to build this yourself vs adopt a module. You can hand-roll the whole hierarchy, but Microsoft ships two heavyweight options, and the decision of when to reach for them matters. The CAF enterprise-scale ALZ module (Azure/caf-enterprise-scale/azurerm) deploys an opinionated entire landing zone — management-group tree, policies, RBAC, and platform subscriptions — in one module; it is powerful and correct but large and opinionated. Azure Verified Modules (AVM) are Microsoft’s newer, supported, per-resource and per-pattern modules: Azure/avm-res-* for single resources (e.g. Azure/avm-res-network-virtualnetwork/azurerm) and Azure/avm-ptn-* for patterns (including an ALZ pattern module). Here is the grid:

Using an AVM resource module is a one-liner swap for your own — same composition, someone else maintains the hardening:

module "vnet" {
  source              = "Azure/avm-res-network-virtualnetwork/azurerm"
  version             = "~> 0.8"
  name                = "vnet-${local.name_prefix}"
  resource_group_name = azurerm_resource_group.this.name
  location            = var.location
  address_space       = var.vnet_address_space
  # AVM modules bundle sensible diagnostic + private-endpoint defaults
}

The Contoso rule of thumb: roll your own while you are learning the estate and it is small; adopt AVM resource/pattern modules for hardened building blocks as you scale; reach for CAF enterprise-scale only for a true greenfield enterprise ALZ where you accept its opinions. Mixing is fine and common — your own composition, AVM leaves.

SRE as code: tags, diagnostics, alerts, RBAC & drift

The difference between infrastructure and a platform is that reliability is a property of every resource, guaranteed by the code, not something SRE bolts on after the first incident. This section is the SRE plane, and it is the densest part of the lesson because there are several controls and each is a small, standard pattern you apply everywhere.

Standardised tags and names. One locals block defines the naming convention and the common_tags map, merged into every resource via tags = local.common_tags. This is what makes cost allocation, ownership and incident routing possible — and a management-group policy (above) requires the tags so nobody can skip them:

# environments/dev/locals.tf
locals {
  environment = "dev"
  workload    = "contoso-platform"
  name_prefix = "${local.workload}-${local.environment}"   # e.g. contoso-platform-dev

  common_tags = {
    environment = local.environment
    workload    = local.workload
    managed_by  = "terraform"
    cost_center = var.cost_center
    owner       = var.owner_email
    repo        = "contoso/landing-zone"
  }
}

Diagnostic settings on every resource. We already saw the network module create its own azurerm_monitor_diagnostic_setting → Log Analytics. That is the pattern everywhere: each module takes a log_analytics_workspace_id and wires its resource’s logs and metrics to it. The monitoring module owns the workspace and the on-call action group:

# modules/monitoring/main.tf
resource "azurerm_log_analytics_workspace" "this" {
  name                = "law-${var.name_prefix}"
  location            = var.location
  resource_group_name = var.resource_group_name
  sku                 = "PerGB2018"
  retention_in_days   = var.retention_days
  daily_quota_gb      = var.daily_quota_gb     # cost guardrail on ingestion
  tags                = var.tags
}

resource "azurerm_monitor_action_group" "oncall" {
  name                = "ag-${var.name_prefix}-oncall"
  resource_group_name = var.resource_group_name
  short_name          = "oncall"
  email_receiver {
    name          = "platform-oncall"
    email_address = var.oncall_email
  }
}

Alerts and budgets. Alerts (azurerm_monitor_metric_alert) fire on resource signals and page the action group; budgets (azurerm_consumption_budget_resource_group) fire on spend. Both are code:

# environments/dev/main.tf (excerpt) — an alert wired to the action group
resource "azurerm_monitor_metric_alert" "storage_availability" {
  name                = "alert-${local.name_prefix}-stg-avail"
  resource_group_name = azurerm_resource_group.this.name
  scopes              = [module.data.storage_account_id]
  description         = "Storage availability dropped below 99%."
  severity            = 1

  criteria {
    metric_namespace = "Microsoft.Storage/storageAccounts"
    metric_name      = "Availability"
    aggregation      = "Average"
    operator         = "LessThan"
    threshold        = 99
  }

  action { action_group_id = module.monitoring.action_group_id }
}

# A cost guardrail: budget with alerts at 80% actual and 100% forecast
resource "azurerm_consumption_budget_resource_group" "this" {
  name              = "budget-${local.name_prefix}"
  resource_group_id = azurerm_resource_group.this.id
  amount            = var.monthly_budget
  time_grain        = "Monthly"

  time_period { start_date = var.budget_start_date }   # first of a month, RFC3339

  notification {
    enabled        = true
    threshold      = 80
    operator       = "GreaterThanOrEqualTo"
    threshold_type = "Actual"
    contact_emails = [var.oncall_email]
  }
  notification {
    enabled        = true
    threshold      = 100
    operator       = "GreaterThanOrEqualTo"
    threshold_type = "Forecasted"
    contact_emails = [var.oncall_email]
  }
}

Least-privilege RBAC. Access is azurerm_role_assignment at the tightest scope that works — a built-in role at a resource group or single resource, never subscription Owner handed out casually. Because it is code, access is reviewable in a PR and diffable over time:

resource "azurerm_role_assignment" "app_reads_blob" {
  scope                = module.data.storage_account_id           # resource scope, not sub
  role_definition_name = "Storage Blob Data Reader"               # data-plane, read-only
  principal_id         = module.compute.principal_id              # the app's managed identity
}

“Least-privilege” is only real if you reach for the narrowest built-in role that works — a data-plane role at a resource beats Contributor at a subscription every time. The catalogue you assign from most:

Drift detection and state hygiene. Drift is when the real world diverges from state — someone hotfixes in the portal, or an Azure-side change mutates a field. The SRE practice is a scheduled terraform plan (per environment) in CI: an empty plan means no drift; a non-empty plan is an alert to investigate and reconcile (re-apply to correct, or import/moved/update code to accept the change). State hygiene is the discipline around it: state is remote, locked, versioned and encrypted (the Storage Account gives you soft-delete/versioning); secrets in state are treated as secret (the blob is access-controlled and AAD-authenticated); and you never hand-edit state — you use terraform state mv, import, and moved {}/removed {} blocks. The blast-radius rule ties it together: one state per environment per layer means a corrupt or lost state costs you one environment’s one layer, never the estate.

Hands-on: build it with Terraform

Now assemble a minimal but real slice of the platform end to end: a dev root that composes three modules — monitoring (Log Analytics + action group), network (VNet + subnet + its diagnostic setting), and data (a hardened Storage Account + its diagnostic setting) — plus a least-privilege role assignment, a budget, and a metric alert, all with standard tags and per-env remote state. This is the composition, the output-passing, the remote state, the tags, a diagnostic setting and the SRE wiring in one runnable stack. ⚠️ This creates real Azure resources; a Storage Account and Log Analytics ingestion cost money. Destroy at the end.

Step 0 — one-time state backend (bootstrap). The backend Storage Account must exist before any root can use it. Create it once by hand (it is the one thing you can’t Terraform-with-Terraform on day one):

az group create -n rg-tfstate-platform -l australiaeast
az storage account create -n sttfstateplatform001 -g rg-tfstate-platform \
  -l australiaeast --sku Standard_LRS --min-tls-version TLS1_2 \
  --allow-blob-public-access false
az storage container create -n tfstate --account-name sttfstateplatform001 --auth-mode login

Step 1 — the module files. Create modules/monitoring, modules/network (shown earlier) and modules/data. The data module hardens the Storage Account and emits its own diagnostic setting and storage_account_id output:

# modules/data/main.tf
resource "azurerm_storage_account" "this" {
  name                            = substr(lower(replace("st${var.name_prefix}", "-", "")), 0, 24)
  location                        = var.location
  resource_group_name             = var.resource_group_name
  account_tier                    = "Standard"
  account_replication_type        = var.replication_type
  min_tls_version                 = "TLS1_2"
  public_network_access_enabled   = false
  allow_nested_items_to_be_public = false
  tags                            = var.tags
}

resource "azurerm_monitor_diagnostic_setting" "storage" {
  name                       = "diag-storage-to-law"
  target_resource_id         = "${azurerm_storage_account.this.id}/blobServices/default"
  log_analytics_workspace_id = var.log_analytics_workspace_id
  enabled_log { category_group = "allLogs" }
  metric      { category = "Transaction" }
}

output "storage_account_id" { value = azurerm_storage_account.this.id }

Step 2 — the dev root composition. This is the heart: it creates the resource group, then composes the three modules, passing monitoring’s workspace ID into network and data, and adds the RBAC/budget/alert SRE resources. (main.tf shown; variables.tf, outputs.tf, locals.tf and versions.tf as earlier.)

# environments/dev/main.tf
resource "azurerm_resource_group" "this" {
  name     = "rg-${local.name_prefix}"
  location = var.location
  tags     = local.common_tags
}

module "monitoring" {
  source              = "../../modules/monitoring"
  name_prefix         = local.name_prefix
  location            = var.location
  resource_group_name = azurerm_resource_group.this.name
  retention_days      = var.log_retention_days
  daily_quota_gb      = var.log_daily_quota_gb
  oncall_email        = var.oncall_email
  tags                = local.common_tags
}

module "network" {
  source                     = "../../modules/network"
  name_prefix                = local.name_prefix
  location                   = var.location
  resource_group_name        = azurerm_resource_group.this.name
  address_space              = var.vnet_address_space
  subnets                    = var.subnets
  log_analytics_workspace_id = module.monitoring.log_analytics_workspace_id   # ← output passing
  tags                       = local.common_tags
}

module "data" {
  source                     = "../../modules/data"
  name_prefix                = local.name_prefix
  location                   = var.location
  resource_group_name        = azurerm_resource_group.this.name
  replication_type           = var.storage_replication_type
  log_analytics_workspace_id = module.monitoring.log_analytics_workspace_id   # ← output passing
  tags                       = local.common_tags
}

resource "azurerm_role_assignment" "platform_reader" {
  scope                = azurerm_resource_group.this.id
  role_definition_name = "Reader"
  principal_id         = var.platform_team_object_id
}
# (+ azurerm_consumption_budget_resource_group and azurerm_monitor_metric_alert as shown above)

Every resource that gets created, which module owns it, and why:

The variables that carry the environment differences — the only things that change between dev and prod, because the module code is identical — make the promotion story concrete:

Step 3 — init, plan, apply for dev. From environments/dev/, initialise with the per-env backend config, then plan and apply:

cd environments/dev
terraform init -backend-config=backend.hcl        # binds state key = dev.terraform.tfstate
terraform fmt -check && terraform validate         # cheap gates first
terraform plan  -out=dev.tfplan
terraform apply dev.tfplan

Representative plan output — note the module addressing and that Terraform ordered monitoring before network/data on its own because of the workspace-ID reference:

Terraform will perform the following actions:

  # azurerm_resource_group.this will be created
  # module.monitoring.azurerm_log_analytics_workspace.this will be created
  # module.monitoring.azurerm_monitor_action_group.oncall will be created
  # module.network.azurerm_virtual_network.this will be created
  # module.network.azurerm_subnet.this["app"] will be created
  # module.network.azurerm_monitor_diagnostic_setting.vnet will be created
  # module.data.azurerm_storage_account.this will be created
  # module.data.azurerm_monitor_diagnostic_setting.storage will be created
  # azurerm_role_assignment.platform_reader will be created
  # azurerm_consumption_budget_resource_group.this will be created
  # azurerm_monitor_metric_alert.storage_availability will be created

Plan: 11 to add, 0 to change, 0 to destroy.

Step 4 — verify. Confirm the platform properties, not just that resources exist: that the tags landed, that the diagnostic setting is wired, and that state went to the right key:

# tags present on the RG?
az group show -n rg-contoso-platform-dev --query tags

# diagnostic setting on the VNet points at the workspace?
az monitor diagnostic-settings list \
  --resource $(terraform output -raw vnet_id) --query "[].name"

# state landed on the dev key?
az storage blob list -c tfstate --account-name sttfstateplatform001 \
  --auth-mode login --query "[].name"    # → dev.terraform.tfstate

Step 5 — promote to uat/prod (conceptual). The identical module code now runs for the next environment; you change directory, init with that env’s backend config (a different key), and apply with that env’s tfvars. Nothing in modules/ or the composition changes:

cd ../uat
terraform init -backend-config=backend.hcl   # key = uat.terraform.tfstate
terraform plan -out=uat.tfplan               # same modules, uat.auto.tfvars values
terraform apply uat.tfplan

In real life the uat and prod applies run in CI behind the approval gate from the promotion table — prod applies the already-approved plan artifact, it does not re-plan.

Step 6 — destroy and clean up. ⚠️ Tear down each environment you stood up, then (optionally) the bootstrap backend:

cd environments/dev && terraform destroy    # repeat per env you applied
# only if you are fully done with the backend:
az group delete -n rg-tfstate-platform --yes --no-wait

Testing the modules: fmt, validate, tflint, checkov, terraform test

A module library that many environments depend on must be tested before it ships, or a bad v1.5.0 breaks every environment that bumps to it. The testing ladder runs cheapest-first, and all of it belongs in CI on every PR to modules/:

terraform fmt -check -recursive        # style: fails if any file isn't canonical
terraform validate                     # syntax + internal consistency (needs init)
tflint --recursive                     # lint: azurerm ruleset — deprecated args, bad names
checkov -d . --framework terraform     # security: public storage, missing TLS, open NSGs
terraform test                         # behaviour: assertions on planned/applied values

Native terraform test is the one that proves behaviour. A *.tftest.hcl file sets inputs and asserts on the result of a plan (no cloud needed) or an apply (real, ephemeral resources). This test proves the network module’s naming and subnet-count contract:

# tests/network.tftest.hcl
variables {
  name_prefix                = "test-net"
  location                   = "australiaeast"
  resource_group_name        = "rg-test"
  address_space              = ["10.99.0.0/16"]
  subnets                    = { app = { address_prefixes = ["10.99.1.0/24"] } }
  log_analytics_workspace_id = "/subscriptions/x/resourceGroups/y/providers/Microsoft.OperationalInsights/workspaces/z"
}

run "vnet_named_by_convention" {
  command = plan
  assert {
    condition     = azurerm_virtual_network.this.name == "vnet-test-net"
    error_message = "VNet name did not follow the vnet-<prefix> convention"
  }
}

run "exactly_one_subnet" {
  command = plan
  assert {
    condition     = length(azurerm_subnet.this) == 1
    error_message = "Expected exactly one subnet from the test inputs"
  }
}

The command = plan tests run without Azure credentials for the assertions on planned values, so they are fast CI gates; use command = apply (against a sandbox subscription) only for the few tests that must observe real API behaviour. Run the security scan (checkov) as a required status check so a module can never merge with a regressed control — a public Storage Account should fail the PR, not the production incident.

Variables, outputs & making it reusable

The whole lesson has been about reuse, so this section is about the last mile: parameterising cleanly and knowing when to stop rolling your own. The environments already share modules and differ only by tfvars — that is the for_each-over-environments idea expressed as folders. If you truly have many near-identical environments, you can even drive them from a single map and for_each a wrapper module, but for a landing zone the explicit folder-per-env is usually clearer and safer than a clever loop, because each env stays independently reviewable and independently applied.

The reuse decision that matters most at scale is your module vs a registry module (AVM). Your own module is right while you are learning the estate, need full control, or have a shape no registry module fits; an AVM resource module is right when you want a hardened, Microsoft-maintained building block and are happy to consume its interface. The tell is maintenance burden: if you find yourself re-implementing security defaults (private endpoints, diagnostic categories, TLS floors) that AVM already ships, swap to AVM and spend your effort on the composition that is actually yours.

Keep the module interface stable even when you swap the implementation: if network exposes vnet_id and subnet_ids, environments consuming it don’t care whether inside it is your azurerm_virtual_network or an AVM module — which is exactly why a good output contract is the most valuable thing in the library.

Common mistakes and troubleshooting

The failures on a platform are different from the failures on a single config — they cluster around cross-module wiring, per-env state, policy denies, and drift across environments. This is the map:

Four gotchas deserve prose because they are the ones that cost hours:

Cross-module dependencies are implicit — trust the graph. New platform engineers reflexively add depends_on between modules “to be safe”. Don’t. When module.network takes module.monitoring.log_analytics_workspace_id, Terraform already knows monitoring must come first; adding depends_on on top can create false ordering or cycles. Wire modules by passing outputs, and let the reference be the dependency. Reserve depends_on for genuinely hidden dependencies (e.g. an RBAC assignment that must exist before a data-plane call the provider makes).

State-per-env is a discipline, not a default. The single most dangerous moment on a platform is terraform init binding to the wrong state. If two environment folders ever init with the same key, they share a state and the second apply will plan to destroy the first’s resources. Make the key explicit per folder (backend.hcl), and make “confirm the key in the plan header” a step in your runbook. In CI, template the key from the environment name so it cannot be wrong by hand.

A policy deny is the guardrail working, not a Terraform bug. When apply fails with RequestDisallowedByPolicy, the platform just stopped a non-compliant resource — a VM in a banned region, an untagged resource group, a public Storage Account. The fix is never to remove the policy; it is to read the named policy in the error and fix the config (add the tag, change the region, set the SKU) and re-plan. This is the entire point of putting guardrails at the management group.

Drift across environments is normal signal — reconcile deliberately. Because each env has its own state, drift shows up per environment, and it is usually one env that got hand-touched (a dev hotfix in the portal that never made it to code, or a prod emergency change). The scheduled terraform plan per env surfaces it; the discipline is to reconcile — either re-apply to bring reality back to code, or bring code up to reality with import/moved/an edit — and to trace who changed it out of band, because unmanaged change is the thing a landing zone exists to eliminate.

Cost, cleanup & production notes

The hands-on slice is cheap but not free. The rough monthly cost if you leave it running, and how to kill it:

The dominant cost is Log Analytics ingestion, which is exactly why daily_quota_gb is set on the workspace and a budget guards the resource group — both are SRE-as-code controls, not afterthoughts. Destroy each environment with terraform destroy in its folder; keep the bootstrap backend unless you are fully finished.

Five production-hardening notes to carry beyond the demo:

1. State is the crown jewels — remote, locked (blob lease), versioned and soft-deleted, AAD-authenticated (use_azuread_auth = true, no storage keys), and access-controlled. One state per env per layer keeps a loss to one blast radius.
2. Separate subscriptions per environment where you can — the hardest blast boundary Azure offers, and it makes the per-env state and per-env policy exceptions natural.
3. Guardrails high, exceptions low — assign deny/audit policies at the management group so every subscription inherits them; put the rare exception at a subscription or RG with an audit trail.
4. Observability is a module property — every module emits its diagnostic setting; a resource with no logs should be impossible to create, not a thing you discover mid-incident.
5. Promote, never hand-edit prod — the same module code walks dev → uat → prod; prod applies an approved plan artifact from CI. The day someone fixes prod in the portal is the day drift begins.

Cheat-sheet

The dense quick-reference for building an Azure landing zone with Terraform.

Core resources

Backend & state (azurerm)

Commands

Registry modules

Interview and exam questions

1. Why put each environment’s state under a different key rather than a different workspace? Directory-per-env with an explicit key makes each environment an independently reviewable folder whose state target is readable in backend.hcl, avoids terraform.workspace interpolation you must reason about under pressure, and lets environments differ structurally when they must. Workspaces share one config and backend — fine for identical short-lived copies, weaker for a governed landing zone.

2. How does a root config order module creation without depends_on? By reference: when module.network uses module.monitoring.log_analytics_workspace_id, Terraform’s dependency graph orders monitoring first automatically. Passing outputs is declaring the dependency; explicit depends_on between modules is usually wrong and can create cycles.

3. What does azurerm_management_group_policy_assignment buy you over per-resource checks? Inheritance and enforcement: a policy assigned at a management group applies to every subscription and resource beneath it, and a deny effect blocks non-compliance at apply time regardless of who applies. It moves a rule from “reviewer’s memory” to “platform-enforced guardrail”.

4. tfvars-per-env vs Terragrunt — when do you switch? Start native (tfvars + partial backend) for ≤3 environments and one team. Switch to Terragrunt when the (env × stack) count makes copy-paste unmanageable, when you need generated per-account backends, or when you want first-class dependency wiring and run-all across many units.

5. What makes observability “a module property” and why does it matter? Each module takes a log_analytics_workspace_id and creates its own azurerm_monitor_diagnostic_setting, so consuming the module is enabling diagnostics. It matters because it makes “a resource with no logs” impossible to create — you never discover missing telemetry mid-incident.

6. Your dev plan proposes to destroy prod resources. Diagnose. The two environments are sharing a state key — dev init’d against prod’s key (or they were never separated). Stop, do not apply, fix the key per environment, re-init, and confirm the plan header points at the right state.

7. An apply fails with RequestDisallowedByPolicy. Is Terraform broken? No — a guardrail worked. A management-group policy denied a non-compliant resource (banned region, missing required tag, public storage). Read the named policy in the error, fix the config, re-plan. Don’t remove the policy.

8. What is the blast-radius argument for separate state per env and a separate platform state? State is the failure domain: corrupting or losing one state costs exactly what that state manages. Per-env state contains a mistake to one env; putting the tenant-wide management-group/policy layer in its own state (and often a different team’s hands) keeps a routine env apply from ever touching org-wide guardrails.

9. (Associate-style) The azurerm backend block references a var for the key. What happens? It fails at init — the backend is read before variables/locals are evaluated, so no interpolation is allowed there. Move the key to partial config: terraform init -backend-config=backend.hcl (or -backend-config="key=dev.terraform.tfstate").

10. (Associate-style) You bumped a module’s version and only prod broke. Why, and the safe process? prod pinned a version with a breaking interface change while dev/uat were still on the old pin (or vice-versa). Pin source and version, upgrade dev first, read the module CHANGELOG, promote the upgrade through uat to prod like any other change — never bump prod’s pin in isolation.

11. When would you adopt Azure/caf-enterprise-scale instead of your own modules? For a greenfield enterprise ALZ where you want the whole opinionated management-group tree, policies and platform subscriptions in one supported module and you accept its opinions. For an existing/small estate, roll your own or compose AVM resource modules — CAF enterprise-scale is a large surface to adopt wholesale.

12. How do you detect and handle drift across a multi-env estate? Run a scheduled terraform plan per environment in CI; an empty plan means no drift, a non-empty plan is an alert. Reconcile deliberately — re-apply to restore code-as-truth, or bring code to reality via import/moved/an edit — and trace the out-of-band change so it stops recurring.

Key takeaways

• A platform is reuse plus isolation. One versioned modules/ library (network/compute/aks/data) consumed by per-environment roots, each with its own remote-state key — that combination is the whole game.
• Compose by passing outputs. The root wires modules (monitoring’s workspace ID into everything, network’s subnets into compute/aks); the reference is the dependency, so skip depends_on.
• Isolate environments deliberately — tfvars-per-env for a small estate, Terragrunt when (env × stack) grows — and promote the same code dev → uat → prod, never hand-editing prod.
• Guardrails high, exceptions low. Management-group Azure Policy (azurerm_management_group_policy_assignment) makes non-compliance fail at apply; subscriptions are your hard blast boundary.
• Adopt AVM/CAF when maintenance outweighs control — hardened building blocks (Azure/avm-res-*) or a full ALZ (Azure/caf-enterprise-scale) — but keep your module interfaces stable when you swap implementations.
• SRE is code, on every resource — standard tags/names, a diagnostic setting to Log Analytics, alerts, budgets, least-privilege azurerm_role_assignment, and a scheduled drift plan.
• Test the library before it ships — fmt → validate → tflint → checkov → terraform test, as required CI gates, so a bad module version can never reach an environment.
• Blast radius is designed in the folder tree — one state per env per layer, the governance plane isolated in its own state, so no single failure or mistake can reach the whole estate.