Terraform Module: Azure Diagnostic Settings — one wrapper to ship every resource's logs and metrics to Log Analytics

Quick take — A reusable hashicorp/azurerm ~> 4.0 module for azurerm_monitor_diagnostic_setting that routes any resource’s platform logs and metrics to Log Analytics, Storage, or Event Hub with allowlist-driven category control. New here? Jump to the Quickstart below to deploy it in minutes; read on for how it works and when to reach for it.

Quickstart (copy-paste)

Minimal, runnable configuration — drop this in a .tf file and fill in the "..." placeholders (each required input is commented):

provider "azurerm" {
  features {}
}

module "diagnostic_setting" {
  source = "git::https://dev.azure.com/teknohut/kloudvin/_git/terraform-modules//terraform-module-azure-diagnostic-setting?ref=v1.0.0"

  name               = "..."  # Name of the diagnostic setting, unique per target resou…
  target_resource_id = "..."  # Full resource ID of the resource to collect telemetry f…
}

Then terraform init && terraform apply. Every other input has a sensible default — see Inputs below to override behaviour.

What this module is

A diagnostic setting is the piece of Azure Monitor plumbing that tells a resource where to send its platform telemetry. Almost every Azure resource emits two kinds of platform telemetry — resource logs (categorised streams like AuditEvent on a Key Vault, kube-apiserver on AKS, or StorageRead on a storage account) and platform metrics — but none of it is collected until you attach an azurerm_monitor_diagnostic_setting that names a destination. Without a diagnostic setting, that data simply ages out of the resource’s short-lived internal buffer and is gone forever.

The catch is that diagnostic settings are per-resource and the available log categories differ for every resource type. Authoring one by hand for a Key Vault, then again for an App Service, then again for a hundred storage accounts means copy-pasting blocks, hard-coding category names that drift between API versions, and inevitably forgetting to enable a category that an auditor later asks for. Wrapping it in a module gives you one tested interface: you pass a target_resource_id and a destination, and the module fans the telemetry out consistently. It also lets you express the common production requirement — “send all log categories but only the metrics I care about” — through the enabled_log_categories / enabled_log_category_groups inputs instead of enumerating every category at every call site.

When to use it

• You are onboarding resources to a central Log Analytics workspace for security monitoring (Microsoft Sentinel), troubleshooting, or compliance, and want every resource configured the same way.
• A policy or audit requirement (PCI-DSS, ISO 27001, an internal CIS baseline) mandates that resource logs for a given service are retained and queryable.
• You need to dual-ship telemetry: cheap long-term retention to a Storage Account for archival, plus Log Analytics or an Event Hub for live querying and SIEM forwarding.
• You manage many resources of the same type (storage accounts, NSGs, SQL databases) and want a for_each loop wrapping one module rather than dozens of bespoke resource blocks.
• You are building an Azure Verified Module-style internal library and want diagnostic settings to be a composable building block other modules can call.

Reach for the platform-native Azure Policy DeployIfNotExists approach instead when you want diagnostic settings forced on all current and future resources in a subscription automatically. This module is for the cases where Terraform owns the resource lifecycle and you want the setting declared alongside the resource in code.

Module structure

terraform-module-azure-diagnostic-setting/
├── versions.tf      # provider + Terraform version pins
├── main.tf          # azurerm_monitor_diagnostic_setting wiring
├── variables.tf     # var-driven inputs with validation
└── outputs.tf       # id / name + destination echoes

versions.tf

terraform {
  required_version = ">= 1.5.0"

  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.0"
    }
  }
}

main.tf

# Discover which log categories and category groups this specific resource
# actually supports, so we can drive an allowlist without hard-coding names.
data "azurerm_monitor_diagnostic_categories" "this" {
  resource_id = var.target_resource_id
}

locals {
  # If no explicit log allowlist is given, fall back to every supported
  # category group ("allLogs" exists on most modern resource types).
  resolved_log_categories = var.enabled_log_categories
  resolved_log_groups = (
    length(var.enabled_log_categories) == 0 && length(var.enabled_log_category_groups) == 0
    ? [for g in data.azurerm_monitor_diagnostic_categories.this.log_category_groups : g if g == "allLogs"]
    : var.enabled_log_category_groups
  )

  # Metrics: enable the requested set, defaulting to "AllMetrics" when asked to.
  resolved_metrics = (
    var.enable_all_metrics
    ? [for m in data.azurerm_monitor_diagnostic_categories.this.metrics : m if m == "AllMetrics"]
    : var.enabled_metric_categories
  )
}

resource "azurerm_monitor_diagnostic_setting" "this" {
  name               = var.name
  target_resource_id = var.target_resource_id

  # Exactly one (or more) of these destinations must be set. They are all
  # optional at the schema level; the variable validation below enforces it.
  log_analytics_workspace_id     = var.log_analytics_workspace_id
  log_analytics_destination_type = var.log_analytics_workspace_id != null ? var.log_analytics_destination_type : null

  storage_account_id = var.storage_account_id

  eventhub_authorization_rule_id = var.eventhub_authorization_rule_id
  eventhub_name                  = var.eventhub_authorization_rule_id != null ? var.eventhub_name : null

  partner_solution_id = var.partner_solution_id

  # Individually named log categories (e.g. "AuditEvent", "StorageRead").
  dynamic "enabled_log" {
    for_each = toset(local.resolved_log_categories)
    content {
      category = enabled_log.value
    }
  }

  # Category groups (e.g. "allLogs", "audit") — convenient broad selectors.
  dynamic "enabled_log" {
    for_each = toset(local.resolved_log_groups)
    content {
      category_group = enabled_log.value
    }
  }

  # Platform metric streams (commonly just "AllMetrics").
  dynamic "enabled_metric" {
    for_each = toset(local.resolved_metrics)
    content {
      category = enabled_metric.value
    }
  }

  lifecycle {
    precondition {
      condition = (
        length(local.resolved_log_categories) +
        length(local.resolved_log_groups) +
        length(local.resolved_metrics)
      ) > 0
      error_message = "At least one log category, log category group, or metric must be enabled for the diagnostic setting."
    }
  }
}

Note on retention: the old retention_policy block inside azurerm_monitor_diagnostic_setting was removed in the azurerm 4.x provider. Storage-account log retention is now managed separately via Azure Monitor data collection / lifecycle, and Log Analytics retention is set on the workspace or per-table. This module deliberately does not expose a retention_policy argument because it no longer exists in the schema.

variables.tf

variable "name" {
  description = "Name of the diagnostic setting (unique per target resource)."
  type        = string

  validation {
    condition     = length(var.name) >= 1 && length(var.name) <= 260
    error_message = "Diagnostic setting name must be between 1 and 260 characters."
  }
}

variable "target_resource_id" {
  description = "Resource ID of the Azure resource whose logs/metrics are collected."
  type        = string

  validation {
    condition     = can(regex("^/subscriptions/", var.target_resource_id))
    error_message = "target_resource_id must be a full Azure resource ID beginning with /subscriptions/."
  }
}

variable "log_analytics_workspace_id" {
  description = "Resource ID of the Log Analytics workspace destination. Set to null if not used."
  type        = string
  default     = null
}

variable "log_analytics_destination_type" {
  description = "How logs land in Log Analytics: 'Dedicated' (resource-specific tables) or 'AzureDiagnostics' (legacy shared table). Ignored if no workspace is set."
  type        = string
  default     = "Dedicated"

  validation {
    condition     = contains(["Dedicated", "AzureDiagnostics"], var.log_analytics_destination_type)
    error_message = "log_analytics_destination_type must be either 'Dedicated' or 'AzureDiagnostics'."
  }
}

variable "storage_account_id" {
  description = "Resource ID of a Storage Account destination for archival. Set to null if not used."
  type        = string
  default     = null
}

variable "eventhub_authorization_rule_id" {
  description = "Authorization rule ID of an Event Hub namespace destination (for SIEM/stream forwarding). Set to null if not used."
  type        = string
  default     = null
}

variable "eventhub_name" {
  description = "Specific Event Hub name to stream to. Required only when eventhub_authorization_rule_id targets a namespace and you want a named hub."
  type        = string
  default     = null
}

variable "partner_solution_id" {
  description = "Resource ID of a partner monitoring solution (e.g. Datadog) destination. Set to null if not used."
  type        = string
  default     = null
}

variable "enabled_log_categories" {
  description = "Explicit list of individual resource log categories to enable (e.g. ['AuditEvent']). Leave empty to use category groups instead."
  type        = list(string)
  default     = []
}

variable "enabled_log_category_groups" {
  description = "List of log category groups to enable (e.g. ['allLogs'] or ['audit']). When both this and enabled_log_categories are empty, the module enables 'allLogs' if the resource supports it."
  type        = list(string)
  default     = []
}

variable "enable_all_metrics" {
  description = "Convenience switch to enable the 'AllMetrics' platform metric category when the resource supports it."
  type        = bool
  default     = true
}

variable "enabled_metric_categories" {
  description = "Explicit list of metric categories to enable. Ignored when enable_all_metrics is true."
  type        = list(string)
  default     = []
}

variable "destinations_required" {
  description = "Guard rail: when true, the module asserts that at least one destination is configured."
  type        = bool
  default     = true

  validation {
    condition     = var.destinations_required == true
    error_message = "destinations_required is a safety flag and must remain true."
  }
}

outputs.tf

output "id" {
  description = "Resource ID of the created diagnostic setting."
  value       = azurerm_monitor_diagnostic_setting.this.id
}

output "name" {
  description = "Name of the diagnostic setting."
  value       = azurerm_monitor_diagnostic_setting.this.name
}

output "target_resource_id" {
  description = "Resource ID of the monitored resource."
  value       = azurerm_monitor_diagnostic_setting.this.target_resource_id
}

output "log_analytics_workspace_id" {
  description = "Log Analytics workspace destination, or null when not used."
  value       = azurerm_monitor_diagnostic_setting.this.log_analytics_workspace_id
}

output "enabled_log_categories" {
  description = "Individual log categories that were enabled on this setting."
  value       = local.resolved_log_categories
}

output "enabled_log_category_groups" {
  description = "Log category groups that were enabled on this setting."
  value       = local.resolved_log_groups
}

output "available_log_category_groups" {
  description = "All log category groups the target resource supports (discovered at plan time) — useful for auditing coverage."
  value       = data.azurerm_monitor_diagnostic_categories.this.log_category_groups
}

How to use it

# A central Log Analytics workspace and an archival storage account already exist.
data "azurerm_log_analytics_workspace" "security" {
  name                = "law-sec-prod-weu"
  resource_group_name = "rg-monitoring-prod"
}

resource "azurerm_key_vault" "app" {
  name                = "kv-payments-prod-weu"
  location            = "westeurope"
  resource_group_name = "rg-payments-prod"
  tenant_id           = data.azurerm_client_config.current.tenant_id
  sku_name            = "standard"
}

# Ship every Key Vault log (incl. AuditEvent) to the security workspace,
# plus AllMetrics, using resource-specific (Dedicated) tables.
module "diagnostic_settings" {
  source = "git::https://dev.azure.com/teknohut/kloudvin/_git/terraform-modules//terraform-module-azure-diagnostic-setting?ref=v1.0.0"

  name                           = "diag-to-sentinel"
  target_resource_id             = azurerm_key_vault.app.id
  log_analytics_workspace_id     = data.azurerm_log_analytics_workspace.security.id
  log_analytics_destination_type = "Dedicated"

  enabled_log_category_groups = ["allLogs"]
  enable_all_metrics          = true
}

# Downstream: alert when the diagnostic setting reports Key Vault auth failures.
# Uses the module's `id` output as the scope so the alert can't be created
# before telemetry is actually flowing.
resource "azurerm_monitor_scheduled_query_rules_alert_v2" "kv_auth_failures" {
  name                = "alert-kv-auth-failures"
  resource_group_name = "rg-monitoring-prod"
  location            = "westeurope"
  scopes              = [data.azurerm_log_analytics_workspace.security.id]
  severity            = 2
  evaluation_frequency = "PT5M"
  window_duration      = "PT15M"

  criteria {
    query                   = <<-KQL
      AzureDiagnostics
      | where ResourceId == "${module.diagnostic_settings.target_resource_id}"
      | where Category == "AuditEvent" and ResultSignature == "Forbidden"
    KQL
    time_aggregation_method = "Count"
    threshold               = 5
    operator                = "GreaterThan"
  }
}

With Terragrunt

Terragrunt keeps this module DRY across environments — define the backend and provider once in a root config, then a thin terragrunt.hcl per environment supplies only the inputs that differ.

1. Root config — live/terragrunt.hcl (inherited by every module):

remote_state {
  backend = "azurerm"
  generate = { path = "backend.tf", if_exists = "overwrite" }
  config = {
    # ...azurerm state bucket/container + key per path...
  }
}

2. Module config — live/prod/diagnostic_setting/terragrunt.hcl:

include "root" {
  path = find_in_parent_folders()
}

terraform {
  source = "git::https://dev.azure.com/teknohut/kloudvin/_git/terraform-modules//terraform-module-azure-diagnostic-setting?ref=v1.0.0"
}

inputs = {
  name = "..."
  target_resource_id = "..."
}

3. Deploy one environment, or roll out all modules together:

cd live/prod/diagnostic_setting && terragrunt apply        # this module
terragrunt run-all apply                      # every module under live/prod

Why Terragrunt here: the backend and provider live in one place instead of being copy-pasted into every module; inputs is overridden per environment (dev / stage / prod) without forking the module; and run-all orchestrates dependencies across modules. Reach for it once you have more than one environment or more than a handful of modules — for a single stack, the plain Quickstart above is enough.

Inputs

Outputs

Enterprise scenario

A financial-services platform team runs Microsoft Sentinel on a single regional Log Analytics workspace and must prove, for PCI-DSS, that audit logs for every Key Vault, SQL database, and storage account in the cardholder-data subscriptions are collected and queryable. They for_each this module over a map of resource IDs in each landing-zone stack, pinned to ?ref=v1.0.0, sending allLogs to the Sentinel workspace and dual-shipping AuditEvent to a WORM-locked archival Storage Account for the seven-year retention clause. Because the module discovers supported categories at plan time, onboarding a new resource type never breaks on an unknown category name, and the available_log_category_groups output feeds a coverage report the compliance team reviews each quarter.

Best practices

• Prefer Dedicated destination type over AzureDiagnostics. Resource-specific tables (e.g. AZKVAuditLogs) have a typed schema, query far cheaper than the giant shared AzureDiagnostics table, and are what modern Sentinel analytics rules expect — only fall back to AzureDiagnostics for resource types that don’t support dedicated tables.
• Use category groups (allLogs/audit) rather than enumerating individual categories. Microsoft adds new log categories to services over time; an allLogs setting picks them up automatically, whereas a hand-listed allowlist silently misses them and leaves a compliance gap.
• Control cost at the destination, not by dropping logs. Verbose categories like storage StorageRead/StorageWrite or AKS kube-audit can dominate Log Analytics ingestion spend — send those to a cheap Storage Account or use Basic/Auxiliary logs tables for high-volume, rarely-queried streams instead of the analytics tier.
• Never store the only copy of audit logs in the same blast radius as the resource. For security-relevant AuditEvent data, ship to a workspace (and ideally an immutable, legal-hold Storage Account) in a separate, tightly-RBAC’d subscription so a compromised resource owner can’t tamper with their own trail.
• Standardise the name and one setting per destination. A resource allows up to five diagnostic settings; give each a stable, descriptive name like diag-to-sentinel or archive-to-storage so Terraform doesn’t churn them and operators can tell destinations apart at a glance.
• Pair this module with Azure Policy DeployIfNotExists for full coverage. Terraform guarantees the setting on resources it manages; a policy assignment catches everything created outside IaC — together they close the “resource with no diagnostic setting” gap that auditors look for.