Terraform Lesson 61 of 89

Terraform on AWS EKS: Installing the AWS Load Balancer Controller (IRSA + Helm)

An EKS cluster on its own cannot put a load balancer in front of your pods the way you want one. Kubernetes has the concept of an Ingress and a Service of type: LoadBalancer, but something has to watch those objects and turn them into real AWS infrastructure — an Application Load Balancer, its listeners, its target groups, its security group — and keep the target group’s members in lock-step with the pods as they come and go. On AWS the modern answer is the AWS Load Balancer Controller (LBC): a controller you run inside the cluster that watches Ingress objects and provisions ALBs, watches annotated Service objects and provisions NLBs, and (via the TargetGroupBinding CRD) can even attach pods to a load balancer you built in Terraform. It has quietly replaced the old in-tree cloud provider, which could only ever stand up a Classic ELB and never understood an Ingress at all.

Installing it is a small but famously fiddly graph of two clouds meeting: an IAM role granted to a Kubernetes ServiceAccount through IRSA, a Helm chart installed by Terraform’s helm provider, and a set of cluster tags the controller reads to discover where to place the ALB. Get any one of the three wrong and the symptom is the same maddening blank — a controller pod in CrashLoopBackOff, or an Ingress whose ADDRESS column never fills in. This lesson builds the whole thing in Terraform, the way you would run it in production: the controller’s IRSA role from the official IAM policy, the kubernetes and helm providers wired from the cluster’s endpoint and token, the helm_release that installs the controller, an IngressClass, and then a test Ingress you apply to watch an ALB appear and route to pod IPs. You will init → plan → apply → verify → destroy, and you will learn the one cleanup rule that saves you a surprise bill: delete your Ingresses before you destroy the controller, or the ALBs it made are orphaned.

This is the provider-specific, hands-on layer of the course. It assumes you know core Terraform — HCL, providers, resources, variables, state and modules — and that an EKS cluster already exists (built in the cluster lesson). The controller’s IRSA role builds directly on the EKS OIDC & IRSA lesson; the kubernetes/helm provider wiring is treated in depth in the Kubernetes & Helm providers lesson; and putting HTTPS, a cert and DNS on top of the Ingress the controller creates is the subject of the EKS Ingress, ACM SSL & ExternalDNS lesson. Here we install the engine that makes all of that possible.

What you’ll build

The scenario is the first thing every EKS team needs after the cluster is up: a way to expose an application on a public HTTP(S) endpoint, backed by a real AWS load balancer, driven by Kubernetes objects rather than click-ops. You already have an EKS cluster (VPC, subnets, node group, an OIDC provider). What’s missing is the piece that translates a Kubernetes Ingress into an ALB. You will install the AWS Load Balancer Controller into kube-system, give it an IRSA role carrying the official controller IAM policy, register an IngressClass named alb, and then apply a tiny demo app + Ingress and watch the controller stand up an internet-facing ALB whose target group is filled with your pods’ IPs. Curl the ALB’s DNS name and you’re serving traffic — with zero load-balancer resources written by hand in the console.

The architecture in words is a chain that crosses from Terraform into the cluster and back out into AWS. Terraform builds two IAM objects (an aws_iam_policy from the controller’s published JSON, and an aws_iam_role whose trust policy federates the cluster’s OIDC provider to one ServiceAccount). Terraform’s helm provider then installs the controller chart, telling it the cluster name, region, VPC id, and the role ARN to annotate onto its ServiceAccount. The controller comes up as a Deployment in kube-system and, from that moment, watches the Kubernetes API for Ingress objects. When you apply one with ingressClassName: alb, the controller calls the ELBv2 API — the same elasticloadbalancing:CreateLoadBalancer you’d call from Terraform — and provisions an ALB, discovering which subnets to use from tags on your VPC subnets, then registers your pod IPs into its target group.

Why Terraform rather than the AWS CLI, eksctl, or raw kubectl + helm? Because this install is exactly the kind of cross-cloud graph Terraform exists to wire: the role ARN produced by the IAM resources must flow into the Helm values; the Helm release must not run until the role and the cluster exist; the IngressClass must not run until the controller’s CRDs are installed. eksctl create iamserviceaccount and a manual helm install do the same job imperatively, in two tools, with no plan, no single source of truth, and no reproducibility across dev/staging/prod. In Terraform it is one apply, one state file, one review-able diff — and the same code stamps the controller into every cluster you own.

Terraform-built EKS load-balancing control plane — Terraform creates the AWS Load Balancer Controller's IRSA role from the official IAM policy, the helm provider installs the aws-load-balancer-controller chart into kube-system and annotates its ServiceAccount with the role ARN, the controller then watches Kubernetes Ingress objects whose class is alb and provisions a real internet-facing Application Load Balancer that routes in target-type=ip mode straight to pod IPs, discovering the public subnets by tag

Reading that diagram left to right is reading the install you’re about to run: Terraform builds the IRSA role (badge 1) and the IAM policy, the helm_release installs the controller (badge 2) into the cluster — a step that only works once the providers are wired from the already-built cluster (badge 3) — and the running controller then reconciles an Ingress into an ALB (badge 4), discovering subnets by tag (badge 5). Badge 6 marks the cleanup trap: the ALB lives outside Terraform state, so an Ingress must be deleted before the controller is destroyed.

Here is the full inventory a single terraform apply adds on top of an existing cluster, and roughly what each costs if you leave it running (Mumbai / ap-south-1, on-demand, indicative July 2026):

Resource (Terraform) AWS / K8s object Role in the build Rough cost if left up
aws_iam_policy IAM policy The official AWSLoadBalancerControllerIAMPolicy Free
aws_iam_role (+ attach) IAM role, OIDC-trusted The controller’s IRSA identity Free
helm_release Deployment, RBAC, CRDs, webhooks The controller itself, in kube-system Free (compute on existing nodes)
kubernetes_ingress_class_v1 IngressClass alb Marks which Ingresses the controller owns Free
kubernetes_deployment_v1 + _service_v1 Demo app + Service The thing behind the Ingress Negligible (pods on existing nodes)
kubernetes_ingress_v1 Ingress alb Triggers the ALB creation
ALB (controller-created) Application Load Balancer Provisioned by the controller, not in TF state ~₹1,400/mo + LCU (~$16+)

The controller, its policy and role are all free — they’re control-plane glue. The line item that bills is the ALB the controller creates when you apply the Ingress, and because that ALB is not in Terraform state, it is the one thing you must remember to clean up in the right order. Every costly or destructive step below is marked ⚠️.

What the AWS Load Balancer Controller actually does

The controller is a Kubernetes controller in the strict sense: a reconciliation loop that watches API objects and drives real-world resources toward the desired state those objects describe. It watches three kinds of object, and produces a different piece of AWS load balancing for each — this table is the mental model to carry through the whole lesson:

It watches With It provisions Terraform-adjacent equivalent
Ingress ingressClassName: alb + alb.ingress.* annotations An Application Load Balancer (L7), listeners, target groups, rules aws_lb (application) + aws_lb_target_group + aws_lb_listener
Service type: LoadBalancer aws-load-balancer-type: external + nlb-target-type A Network Load Balancer (L4) aws_lb (network)
TargetGroupBinding (CRD) A target-group ARN + a Service Keeps an existing target group’s members in sync with pods You build the aws_lb/aws_lb_target_group; the controller fills it

Those three modes matter because they cover the whole spectrum of “who owns the load balancer.” With Ingress and Service, the controller owns the ALB/NLB — it creates and deletes it as the object comes and goes. With TargetGroupBinding, Terraform owns the load balancer and target group (you write the aws_lb and aws_lb_target_group yourself), and the controller only owns the membership — it registers and deregisters pods. That last mode is the bridge for teams that want their edge in Terraform/GitOps but still want pods attached automatically.

The controller’s full feature surface is broad; these are the capabilities you actually reach for, and the annotation or CRD that unlocks each:

Capability How you invoke it Notes
L7 ALB from an Ingress ingressClassName: alb The headline feature
Merge many Ingresses onto one ALB alb.ingress.kubernetes.io/group.name IngressGroups share an ALB (cost saving)
TLS from ACM alb.ingress.kubernetes.io/certificate-arn (or auto-discover) Terminate HTTPS at the ALB
HTTP→HTTPS redirect alb.ingress.kubernetes.io/ssl-redirect: '443' Standard secure default
Target-type IP (pods direct) alb.ingress.kubernetes.io/target-type: ip ALB → pod IP, no NodePort hop
WAF / Shield alb.ingress.kubernetes.io/wafv2-acl-arn, shield-advanced-protection L7 protection at the edge
NLB from a Service service.beta.kubernetes.io/aws-load-balancer-type: external L4 alternative to Ingress
Bind an existing target group TargetGroupBinding CRD Terraform owns the LB; controller owns the targets

The NLB path: a Service, not an Ingress

Not everything is L7. When you need an L4 Network Load Balancer — raw TCP/UDP, static IPs, the lowest latency — you don’t write an Ingress; you annotate a Service of type: LoadBalancer so the controller (not the in-tree provider) handles it. The type: external annotation is the switch that hands the Service to the LBC:

Service annotation Value Effect
service.beta.kubernetes.io/aws-load-balancer-type external Hand the Service to the LBC (not the in-tree CLB)
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type ip / instance NLB target mode (ip = pods direct)
service.beta.kubernetes.io/aws-load-balancer-scheme internet-facing / internal Public vs private NLB

Without aws-load-balancer-type: external, a type: LoadBalancer Service falls through to the legacy in-tree provider and you get a Classic ELB — the usual surprise when someone expects an NLB and gets a CLB.

Versus the legacy in-tree cloud provider

Before this controller, Kubernetes-on-AWS relied on the in-tree AWS cloud provider baked into kube-controller-manager. It could do exactly one load-balancing thing: watch Service type: LoadBalancer and create a Classic Load Balancer (CLB) — the legacy box. It had no Ingress support whatsoever, no ALB, no target-type IP, no WAF, no CRDs. The community “ALB Ingress Controller” grew up to fill that gap and, once mature, was renamed the AWS Load Balancer Controller and given NLB duty too. The difference is stark:

Aspect In-tree cloud provider AWS Load Balancer Controller
Runs where Inside kube-controller-manager (control plane) As a Deployment you install (kube-system)
Ingress → ALB Not supported Yes (core feature)
Service → LB CLB only NLB (type: external)
Target type IP (pod-direct) No Yes
ACM, WAF, redirects, IngressGroups No Yes
CRDs (TargetGroupBinding, IngressClassParams) No Yes
AWS credentials Node instance profile (broad) IRSA (scoped to one SA)
Status Deprecated / being removed Current, recommended

The credential story is the other big upgrade. The in-tree provider used the node’s instance profile, which meant every pod on the node inherited load-balancer-creating permissions. The LBC uses IRSA, so only the controller’s ServiceAccount can assume the role — least privilege, and the reason the install starts with an IAM role rather than a node policy.

The IRSA role: giving the controller its AWS permissions

The controller runs as a pod, but it calls AWS APIs — elasticloadbalancing:CreateLoadBalancer, ec2:DescribeSubnets, acm:ListCertificates, dozens more. A pod cannot use an instance profile without inheriting the whole node’s permissions, so we use IRSA (IAM Roles for Service Accounts): the cluster has an OIDC identity provider, and an IAM role’s trust policy federates that provider so that a specific ServiceAccountkube-system:aws-load-balancer-controller — can call sts:AssumeRoleWithWebIdentity and receive scoped, short-lived credentials. If IRSA is new to you, the EKS OIDC & IRSA lesson builds the OIDC provider and the trust anatomy from scratch; here we consume that provider and attach the controller’s specific policy.

Two moving parts make up the role: the permission policy (what the controller may do) and the trust policy (who may assume the role). The trust policy is IRSA boilerplate keyed to the SA; the permission policy is a large, official document AWS publishes alongside each controller release.

The permission policy — use the official JSON, don’t hand-write it

The controller’s IAM policy is big (100+ actions) and it changes between controller versions as features are added. Never hand-roll it. AWS publishes the exact JSON in the controller’s repository, tagged per version. You download the file for the version you’re installing and feed it straight into aws_iam_policy:

# Pin the tag to the controller version you will install (chart appVersion).
curl -sSo iam_policy.json \
  https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.11.0/docs/install/iam_policy.json
# iam.tf — the permission policy, straight from the official file
resource "aws_iam_policy" "lbc" {
  name        = "AWSLoadBalancerControllerIAMPolicy"
  description = "Official policy for the AWS Load Balancer Controller (v2.11.0)"
  policy      = file("${path.module}/iam_policy.json")
}

Conceptually the policy grants a handful of permission families; you don’t memorise the actions, but you should recognise what each family is for when you read a denial in the logs:

Permission family (in the policy) Example actions Why the controller needs it
Describe the network ec2:DescribeSubnets, DescribeVpcs, DescribeSecurityGroups Find subnets (by tag) and VPC to place the LB
Manage security groups ec2:CreateSecurityGroup, AuthorizeSecurityGroupIngress The ALB’s managed SG and its rules
Create/modify ELBv2 elasticloadbalancing:CreateLoadBalancer, CreateTargetGroup, CreateListener, RegisterTargets Build the ALB/NLB and register pods
Read ACM & WAF & Shield acm:ListCertificates, wafv2:*AssociateWebACL, shield:* TLS certs and edge protection
Tag & protect its own resources elasticloadbalancing:AddTags, condition keys on elbv2.k8s.aws/cluster Tag LBs it owns; refuse to touch ones it doesn’t
Read IAM service-linked role iam:CreateServiceLinkedRole (conditioned) The ELB service-linked role, first time

A subtlety worth knowing: newer policy versions condition many write actions on a resource tag (elbv2.k8s.aws/cluster) the controller stamps on everything it creates, so even with the policy attached the controller can only modify load balancers it made — a nice guardrail against it clobbering a Terraform-owned ALB.

The trust policy — module or hand-rolled OIDC

With the permission policy created, you need the role and its OIDC trust. There are two idiomatic ways, and both are worth knowing.

Option A — the iam-role-for-service-accounts-eks module (recommended). The terraform-aws-modules/iam/aws collection ships a submodule that knows the controller by name: set one boolean and it attaches the correct policy and builds the OIDC trust scoped to the SA. This is the least-error-prone path and the one most teams use:

# iam.tf — Option A: the IRSA module does the policy AND the trust
module "lbc_irsa" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version = "~> 5.44"

  role_name                              = "eks-aws-lb-controller"
  attach_load_balancer_controller_policy = true   # attaches the official policy for you

  oidc_providers = {
    main = {
      provider_arn               = var.oidc_provider_arn   # from the cluster/EKS module
      namespace_service_accounts = ["kube-system:aws-load-balancer-controller"]
    }
  }
}
# module.lbc_irsa.iam_role_arn is the value you feed to Helm.

With Option A you can delete the aws_iam_policy + curl step entirely — the module maintains the policy internally per its version. The trade-off is you trust the module’s policy to track the chart version you pin.

Option B — hand-rolled role + OIDC trust. When you want to see (or audit) every line, build the trust document yourself. This is the exact IRSA pattern: a Federated principal of the cluster’s OIDC provider ARN, and two StringEquals conditions pinning the :sub (the ServiceAccount) and the :aud (sts.amazonaws.com):

# iam.tf — Option B: hand-rolled trust, attach the official policy from Option's file
data "aws_iam_policy_document" "lbc_assume" {
  statement {
    effect  = "Allow"
    actions = ["sts:AssumeRoleWithWebIdentity"]
    principals {
      type        = "Federated"
      identifiers = [var.oidc_provider_arn]
    }
    condition {
      test     = "StringEquals"
      variable = "${var.oidc_provider}:sub"
      values   = ["system:serviceaccount:kube-system:aws-load-balancer-controller"]
    }
    condition {
      test     = "StringEquals"
      variable = "${var.oidc_provider}:aud"
      values   = ["sts.amazonaws.com"]
    }
  }
}

resource "aws_iam_role" "lbc" {
  name               = "eks-aws-lb-controller"
  assume_role_policy = data.aws_iam_policy_document.lbc_assume.json
}

resource "aws_iam_role_policy_attachment" "lbc" {
  role       = aws_iam_role.lbc.name
  policy_arn = aws_iam_policy.lbc.arn   # the file()-based policy from above
}
# aws_iam_role.lbc.arn is the value you feed to Helm.

Here var.oidc_provider is the issuer host without the https:// (e.g. oidc.eks.ap-south-1.amazonaws.com/id/EXAMPLE) and var.oidc_provider_arn is the full arn:aws:iam::…:oidc-provider/…. The two approaches compare like this:

Option A — IRSA module Option B — hand-rolled
Lines of HCL ~10 ~35 + the policy file
Policy source Module-maintained The official iam_policy.json you pin
Trust document Built for you You write and can audit it
Version coupling Module version ↔ policy You pin the policy tag yourself
Best for Most teams, fast + correct Audits, air-gapped, custom conditions

The one non-negotiable, whichever you pick: the :sub condition must be exactly system:serviceaccount:kube-system:aws-load-balancer-controller. A namespace or SA-name typo here is the number-one cause of a controller that installs cleanly and then CrashLoopBackOffs with AccessDenied — the pod assumes no role because the trust doesn’t match its identity.

Wiring the kubernetes and helm providers

To install anything into the cluster, Terraform’s kubernetes and helm providers must authenticate to the API server. They need three things: the API endpoint, the cluster CA certificate, and a bearer token. All three are outputs of the cluster you already built — and that is the source of the single biggest gotcha in EKS-with-Terraform, which we’ll hit head-on.

Provider needs Source (cluster output / data source) In the demo
API endpoint cluster_endpoint / data.aws_eks_cluster.this.endpoint var.cluster_endpoint
Cluster CA cert certificate_authority[0].data (base64) base64decode(var.cluster_ca_data)
Bearer token exec (aws eks get-token) or aws_eks_cluster_auth.token The exec plugin

There are two ways to supply the token, and the choice matters:

# providers.tf — the exec plugin (recommended): a fresh token per API call
provider "kubernetes" {
  host                   = var.cluster_endpoint
  cluster_ca_certificate = base64decode(var.cluster_ca_data)

  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "aws"
    args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
  }
}

provider "helm" {
  kubernetes {
    host                   = var.cluster_endpoint
    cluster_ca_certificate = base64decode(var.cluster_ca_data)
    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      command     = "aws"
      args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
    }
  }
}

The alternative uses the aws_eks_cluster_auth data source, which fetches a token at plan/apply time:

data "aws_eks_cluster_auth" "this" {
  name = var.cluster_name
}

provider "kubernetes" {
  host                   = var.cluster_endpoint
  cluster_ca_certificate = base64decode(var.cluster_ca_data)
  token                  = data.aws_eks_cluster_auth.this.token   # expires in ~15 min
}

They differ in exactly one important way — token freshness — which is why the exec form is preferred for anything but the quickest apply:

exec plugin aws_eks_cluster_auth data source
Token fetched On every API call, by the aws CLI Once, at plan/refresh time
Expiry risk None (always fresh) ~15 min — a long apply can fail mid-run
Requires on the runner aws CLI on PATH Nothing extra
CI friendliness Needs the CLI in the image Works with just the provider
Recommended for Long applies, Helm installs, prod Short, simple applies

⚠️ The provider-after-cluster ordering trap

Here is the gotcha, stated plainly: a Terraform provider is configured before any resource is applied — so if the kubernetes/helm provider is configured from attributes of an EKS cluster that does not exist yet, the very first apply fails. On a cold run where one root module builds both the cluster and the controller, Terraform tries to configure the helm provider from cluster_endpoint, gets an unknown/empty value (the cluster isn’t built), and either errors with Kubernetes cluster unreachable or produces an invalid provider config. This is not a bug you can annotate away with depends_on — providers don’t take depends_on.

The robust patterns, in order of preference:

Pattern How When to use
Two root modules cluster/ applies first (VPC, EKS, OIDC); platform/ reads it via remote_state/data sources and installs the controller Production — clean, no ordering hacks
Two-phase apply One module, but terraform apply -target=module.eks first, then a full apply Quick demos, single module
data sources not module outputs Configure providers from data.aws_eks_cluster / data.aws_eks_cluster_auth (which read the live cluster) rather than resource attributes Reduces, doesn’t eliminate, the cold-start problem

This lesson’s demo assumes the cluster already exists and this root module only installs the controller — the cleanest separation, and the one that mirrors the two-module production layout. We read the cluster through variables/data sources, never build it here. The Kubernetes & Helm providers lesson drills into this ordering problem and the two-module layout in full.

Installing the controller with helm_release

With the role built and the providers wired, the install itself is one resource: a helm_release of the aws-load-balancer-controller chart from AWS’s eks-charts repository. The chart bundles everything — the Deployment, its RBAC, the CRDs (IngressClassParams, TargetGroupBinding), and the mutating/validating admission webhooks the controller uses to default and validate Ingress objects.

The chart installs Kind Purpose
aws-load-balancer-controller Deployment The controller pods (leader-elected)
Cluster/Role bindings + ServiceAccount RBAC Lets the controller watch Ingress/Service/pods
IngressClassParams, TargetGroupBinding CRDs Class-wide defaults; bind an existing target group
aws-load-balancer-webhook-service Service + webhooks Mutating/validating admission on Ingress/Service
# helm.tf
resource "helm_release" "lbc" {
  name       = "aws-load-balancer-controller"
  repository = "https://aws.github.io/eks-charts"
  chart      = "aws-load-balancer-controller"
  version    = "1.11.0"          # chart version; appVersion is controller v2.11.0
  namespace  = "kube-system"

  # --- required identity + placement values ---
  set {
    name  = "clusterName"
    value = var.cluster_name
  }
  set {
    name  = "region"
    value = var.region
  }
  set {
    name  = "vpcId"
    value = var.vpc_id
  }

  # --- ServiceAccount: create it here and annotate with the IRSA role ---
  set {
    name  = "serviceAccount.create"
    value = "true"
  }
  set {
    name  = "serviceAccount.name"
    value = "aws-load-balancer-controller"
  }
  set {
    name  = "serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn"
    value = module.lbc_irsa.iam_role_arn      # or aws_iam_role.lbc.arn (Option B)
  }

  # --- HA + resilience ---
  set {
    name  = "replicaCount"
    value = "2"
  }

  depends_on = [module.lbc_irsa]  # role must exist before the SA references it
}

The values you set are the whole contract between Kubernetes and AWS — miss one and the controller either won’t start or won’t find the network:

Helm value What it does Miss it and…
clusterName Names the cluster the controller manages; used in resource tags Required — pod won’t start without it
region AWS region for API calls Auto-detected on EKS, but set it explicitly
vpcId VPC to build load balancers in Auto-detected, but explicit avoids IMDS lookups
serviceAccount.create true = chart makes the SA; false = use an existing one Duplicate/absent SA if it disagrees with reality
serviceAccount.name The SA name (must match the IRSA :sub) Trust mismatch → AccessDenied
serviceAccount.annotations.eks\.amazonaws\.com/role-arn Binds the SA to the IRSA role No AWS credsCrashLoopBackOff
replicaCount Controller replicas (leader-elected) Single point of failure at 1
image.tag / chart version Pins the controller image Floating version drifts behaviour

Two details in that block trip everyone at least once. First, the escaped dots in serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn: Helm’s --set treats . as a path separator, so the literal dots in the annotation key must be escaped, and in HCL each backslash is itself doubled — hence \\.. Get it wrong and the annotation silently lands under the wrong key, the SA has no role ARN, and you’re back to CrashLoopBackOff. Second, the depends_on: the ServiceAccount references the role ARN, so the role must exist first; the explicit dependency guarantees ordering even though the ARN flows through a set value.

Create the ServiceAccount here, or separately?

The block above lets the chart create the ServiceAccount (serviceAccount.create = true). The alternative is to create the SA yourself with kubernetes_service_account_v1 and tell the chart to reuse it (serviceAccount.create = false). Both are valid; the trade-offs:

Approach Config Pros / cons
Chart creates the SA (create = true) Annotate via Helm set (escaped dots) Fewer resources; the escaping is fiddly
Terraform creates the SA (create = false) kubernetes_service_account_v1 with clean annotation map Clean annotations, SA visible in state; one more resource + a depends_on

For the Terraform-managed SA, it’s a small resource with a normal map (no escaping):

resource "kubernetes_service_account_v1" "lbc" {
  metadata {
    name      = "aws-load-balancer-controller"
    namespace = "kube-system"
    annotations = {
      "eks.amazonaws.com/role-arn" = module.lbc_irsa.iam_role_arn
    }
  }
}
# then in helm_release: serviceAccount.create=false, serviceAccount.name=aws-load-balancer-controller

Pin the chart version deliberately. The chart version and the controller image appVersion track closely (chart 1.11.0 ships controller v2.11.0); pinning both means a plan in CI never silently upgrades the controller — and, crucially, the IAM policy you attached must match the controller version, because newer controllers call newer actions. Upgrade the policy and the chart together, never one alone.

IngressClass, target-type, and subnet discovery

The controller is installed, but it won’t touch an Ingress until two things are true: the Ingress names an IngressClass the controller owns, and the controller can discover which subnets to place the ALB in. These are the two most common “I applied an Ingress and nothing happened” causes.

IngressClass and IngressClassParams

An IngressClass is the Kubernetes object that says “Ingresses of this class are handled by this controller.” You register one whose controller field is the LBC’s well-known name, and (optionally) mark it the cluster default:

# ingressclass.tf
resource "kubernetes_ingress_class_v1" "alb" {
  metadata {
    name = "alb"
    annotations = {
      "ingressclass.kubernetes.io/is-default-class" = "true"  # optional: default for classless Ingresses
    }
  }
  spec {
    controller = "eks.amazonaws.com/aws-load-balancer-controller"
  }
  depends_on = [helm_release.lbc]   # the controller/CRDs must be installed first
}

For cluster-wide defaults — a default scheme, default tags, a fixed subnet list, a shared IngressGroup — you attach an IngressClassParams (a CRD the chart installs) to the class, so you don’t repeat annotations on every Ingress:

Object Scope Sets Typical use
IngressClass Which controller owns the Ingress controller, default flag, optional parameters Always — one alb class
IngressClassParams (CRD) Cluster-wide defaults for that class scheme, subnets, tags, group, ipAddressType Enforce internal-by-default, org tags, fixed subnets
alb.ingress.* annotations Per-Ingress overrides Anything above, plus cert-arn, healthcheck, listen-ports The knob you turn per app

Because IngressClassParams is a CRD, managing it with kubernetes_manifest reintroduces the provider-after-cluster problem (that resource needs the CRD’s API schema available at plan time). For the demo we keep the IngressClass plain and set behaviour with per-Ingress annotations; in production, apply the IngressClassParams after the CRDs exist (a later apply, or GitOps).

target-type: ip vs instance

Every ALB the controller builds routes to a target group, and the target group’s target-type decides what the ALB registers — and it’s the most consequential annotation you’ll set:

target-type ALB registers Traffic path Requires Use when
ip Pod IPs directly ALB → pod ENI (one hop) VPC CNI (pods have VPC IPs) Default choice on EKS — lower latency, no NodePort, works with Fargate
instance Node instance + NodePort ALB → node:NodePort → kube-proxy → pod A NodePort Service Custom CNI without routable pod IPs; you need node-level routing

On EKS the VPC CNI gives every pod a real VPC IP address, so the ALB can send traffic straight to the pod’s IP — no NodePort, no extra kube-proxy hop, and it works on Fargate where there are no nodes to NodePort at all. That’s why target-type: ip is the near-universal default on EKS, and the mode our demo uses. Reach for instance only with a CNI that doesn’t give pods routable addresses.

Subnet auto-discovery by tag

The controller has to decide which subnets to build the ALB in. It does this by reading tags on your subnets — this is the single most common reason a first Ingress produces no ALB. Tag the subnets in the same Terraform that builds the VPC:

Tag Value Put it on Effect
kubernetes.io/role/elb 1 Public subnets Where internet-facing ALBs go
kubernetes.io/role/internal-elb 1 Private subnets Where internal ALBs go
kubernetes.io/cluster/<cluster-name> owned or shared All cluster subnets Associates the subnet with the cluster

If those tags are missing, the controller logs unable to discover at least one subnet and the Ingress ADDRESS never fills in. You can bypass discovery by naming subnets explicitly with the alb.ingress.kubernetes.io/subnets annotation, but tagging is the clean, cluster-wide way — and it’s a two-line addition to the subnet resources you already own:

resource "aws_subnet" "public" {
  # ...
  tags = {
    "kubernetes.io/role/elb"                    = "1"
    "kubernetes.io/cluster/${var.cluster_name}" = "shared"
  }
}

Hands-on: build it with Terraform

⚠️ This installs the controller into a real EKS cluster and — at the Ingress step — provisions a real, billable Application Load Balancer. It assumes an existing EKS cluster (endpoint, CA, OIDC provider) and the aws CLI + kubectl + helm on your machine. Follow it end to end and run the destroy step in the right order.

We install the controller, register the IngressClass, and deploy a tiny app + Ingress to watch an ALB appear. Lay out the files:

mkdir -p lbc-demo && cd lbc-demo
touch versions.tf providers.tf variables.tf iam.tf helm.tf \
      ingressclass.tf demo-app.tf outputs.tf
curl -sSo iam_policy.json \
  https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.11.0/docs/install/iam_policy.json

1. Pin Terraform and the providers (versions.tf). Four providers: aws (the IAM role), helm (the release), kubernetes (IngressClass, app, Ingress). Pin each with ~>, and use a remote backend — for AWS that’s S3 for state plus a DynamoDB table for locking:

# versions.tf
terraform {
  required_version = ">= 1.6.0"

  required_providers {
    aws        = { source = "hashicorp/aws",        version = "~> 5.60" }
    helm       = { source = "hashicorp/helm",       version = "~> 2.17" }
    kubernetes = { source = "hashicorp/kubernetes", version = "~> 2.32" }
  }

  backend "s3" {
    bucket         = "kv-tfstate-2026"
    key            = "eks/lbc-demo/terraform.tfstate"
    region         = "ap-south-1"
    dynamodb_table = "kv-tf-locks"
    encrypt        = true
  }
}

2. Variables (variables.tf). The cluster’s identity and networking come in as variables (in a two-module layout these are remote_state outputs from the cluster module). We take the endpoint, CA, name, region, VPC id and OIDC details:

# variables.tf
variable "region"            {
  type = string
  default = "ap-south-1"
}
variable "cluster_name"      { type = string }
variable "cluster_endpoint"  { type = string }
variable "cluster_ca_data"   {
  type = string
  description = "base64 CA (…certificate_authority[0].data)"
}
variable "vpc_id"            { type = string }
variable "oidc_provider_arn" {
  type = string
  description = "arn:aws:iam::…:oidc-provider/oidc.eks…"
}
variable "oidc_provider"     {
  type = string
  description = "oidc.eks.<region>.amazonaws.com/id/… (no https://)"
}
variable "lbc_chart_version" {
  type = string
  default = "1.11.0"
}

3. Providers (providers.tf). The aws provider, plus kubernetes and helm wired from the cluster via the exec plugin (fresh tokens, no 15-minute expiry mid-apply):

# providers.tf
provider "aws" {
  region = var.region
  default_tags { tags = { project = "tf-course", lesson = "eks-lbc", owner = "vinod" } }
}

provider "kubernetes" {
  host                   = var.cluster_endpoint
  cluster_ca_certificate = base64decode(var.cluster_ca_data)
  exec {
    api_version = "client.authentication.k8s.io/v1beta1"
    command     = "aws"
    args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
  }
}

provider "helm" {
  kubernetes {
    host                   = var.cluster_endpoint
    cluster_ca_certificate = base64decode(var.cluster_ca_data)
    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      command     = "aws"
      args        = ["eks", "get-token", "--cluster-name", var.cluster_name]
    }
  }
}

4. The IRSA role (iam.tf). We use the module (Option A) for brevity — it attaches the official policy and builds the OIDC trust scoped to the SA:

# iam.tf
module "lbc_irsa" {
  source  = "terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks"
  version = "~> 5.44"

  role_name                              = "${var.cluster_name}-aws-lb-controller"
  attach_load_balancer_controller_policy = true

  oidc_providers = {
    main = {
      provider_arn               = var.oidc_provider_arn
      namespace_service_accounts = ["kube-system:aws-load-balancer-controller"]
    }
  }
}

(Prefer to see every action? Swap in the Option B aws_iam_policy + aws_iam_role + data.aws_iam_policy_document from the IRSA section, using the iam_policy.json you curled.)

5. The controller (helm.tf). The helm_release, with the required values and the SA annotated with the role ARN:

# helm.tf
resource "helm_release" "lbc" {
  name       = "aws-load-balancer-controller"
  repository = "https://aws.github.io/eks-charts"
  chart      = "aws-load-balancer-controller"
  version    = var.lbc_chart_version
  namespace  = "kube-system"

  set{
    name = "clusterName"
    value = var.cluster_name
  }
  set{
    name = "region"
    value = var.region
  }
  set{
    name = "vpcId"
    value = var.vpc_id
  }
  set{
    name = "replicaCount"
    value = "2"
  }

  set{
    name = "serviceAccount.create"
    value = "true"
  }
  set{
    name = "serviceAccount.name"
    value = "aws-load-balancer-controller"
  }
  set {
    name  = "serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn"
    value = module.lbc_irsa.iam_role_arn
  }

  depends_on = [module.lbc_irsa]
}

6. The IngressClass (ingressclass.tf). Register the alb class the controller owns, after the release installs the CRDs:

# ingressclass.tf
resource "kubernetes_ingress_class_v1" "alb" {
  metadata {
    name        = "alb"
    annotations = { "ingressclass.kubernetes.io/is-default-class" = "true" }
  }
  spec { controller = "eks.amazonaws.com/aws-load-balancer-controller" }
  depends_on = [helm_release.lbc]
}

7. The demo app + Ingress (demo-app.tf). A trivial echoserver Deployment, a ClusterIP Service, and an Ingress in alb class with target-type: ip. Keeping the Ingress as a Terraform resource with depends_on the release is what makes destroy clean up the ALB in the right order:

# demo-app.tf
resource "kubernetes_deployment_v1" "demo" {
  metadata{
    name = "demo"
    namespace = "default"
  }
  spec {
    replicas = 2
    selector { match_labels = { app = "demo" } }
    template {
      metadata { labels = { app = "demo" } }
      spec {
        container {
          name  = "app"
          image = "registry.k8s.io/e2e-test-images/echoserver:2.5"
          port { container_port = 8080 }
        }
      }
    }
  }
}

resource "kubernetes_service_v1" "demo" {
  metadata{
    name = "demo"
    namespace = "default"
  }
  spec {
    selector = { app = "demo" }
    port {
      port        = 80
      target_port = 8080
    }
    type = "ClusterIP"   # target-type=ip doesn't need NodePort
  }
}

resource "kubernetes_ingress_v1" "demo" {
  metadata {
    name      = "demo"
    namespace = "default"
    annotations = {
      "alb.ingress.kubernetes.io/scheme"      = "internet-facing"
      "alb.ingress.kubernetes.io/target-type" = "ip"
    }
  }
  spec {
    ingress_class_name = "alb"
    rule {
      http {
        path {
          path      = "/"
          path_type = "Prefix"
          backend {
            service {
              name = kubernetes_service_v1.demo.metadata[0].name
              port { number = 80 }
            }
          }
        }
      }
    }
  }
  depends_on = [helm_release.lbc, kubernetes_ingress_class_v1.alb]
}

8. Outputs (outputs.tf). Surface the role ARN and the Ingress hostname (the ALB DNS name the controller writes back into the Ingress status):

# outputs.tf
output "lbc_role_arn" { value = module.lbc_irsa.iam_role_arn }
output "ingress_hostname" {
  value       = try(kubernetes_ingress_v1.demo.status[0].load_balancer[0].ingress[0].hostname, "pending…")
  description = "The ALB DNS name the controller provisions (populated a minute after apply)."
}

9. Init. Downloads all four providers and wires the backend:

terraform init
# Initializing provider plugins...
# - Installing hashicorp/aws v5.6x...
# - Installing hashicorp/helm v2.17...
# - Installing hashicorp/kubernetes v2.32...
# Terraform has been successfully initialized!

10. Plan. Pass the cluster’s coordinates (from the cluster module’s outputs) and read the summary — it must add the IAM role, the release, the class and the app:

export TF_VAR_cluster_name="kv-eks"
export TF_VAR_cluster_endpoint="https://ABCD.gr7.ap-south-1.eks.amazonaws.com"
export TF_VAR_cluster_ca_data="LS0tLS1CRUdJTi…"
export TF_VAR_vpc_id="vpc-0abc123"
export TF_VAR_oidc_provider_arn="arn:aws:iam::123456789012:oidc-provider/oidc.eks.ap-south-1.amazonaws.com/id/ABCD"
export TF_VAR_oidc_provider="oidc.eks.ap-south-1.amazonaws.com/id/ABCD"

terraform plan
# Plan: 8 to add, 0 to change, 0 to destroy.

11. Apply. ⚠️ The ALB (billing) is created at the Ingress step. The slow parts are the Helm install (the controller pod pulling its image and passing its webhook readiness) and, ~60–90s later, the ALB provisioning:

terraform apply -auto-approve
# module.lbc_irsa...: Creation complete after 6s
# helm_release.lbc: Still creating... [30s elapsed]
# helm_release.lbc: Creation complete after 48s
# kubernetes_ingress_class_v1.alb: Creation complete after 1s
# kubernetes_deployment_v1.demo: Creation complete after 12s
# kubernetes_ingress_v1.demo: Creation complete after 3s
# Apply complete! Resources: 8 added, 0 changed, 0 destroyed.
# Outputs:
# ingress_hostname = "pending…"   # ALB is provisioning; check again in ~90s

12. Verify — controller healthy, ALB created, traffic served. First confirm the controller Deployment is up, then watch the Ingress get an address, then curl it:

# a) The controller is running (the brief's canonical check):
kubectl -n kube-system get deploy aws-load-balancer-controller
# NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
# aws-load-balancer-controller   2/2     2            2           2m

# b) The Ingress now has an ALB hostname in ADDRESS:
kubectl get ingress demo
# NAME   CLASS   HOSTS   ADDRESS                                              PORTS   AGE
# demo   alb     *       k8s-default-demo-abc123-45678.ap-south-1.elb.amazonaws.com   80    90s

# c) The controller's log shows the reconcile (no errors):
kubectl -n kube-system logs deploy/aws-load-balancer-controller | grep -i "successfully" | tail -2
# "successfully reconciled" ingress default/demo

# d) The ALB exists in AWS, and serves:
ALB=$(kubectl get ingress demo -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
curl -s "http://$ALB/" | head -n1
# CLIENT VALUES:  (echoserver responds → the ALB routes to pods)

The verification checklist:

Step Command Expect
Controller up kubectl -n kube-system get deploy aws-load-balancer-controller 2/2 ready
SA has role ARN kubectl -n kube-system get sa aws-load-balancer-controller -o yaml | grep role-arn Your IRSA role ARN
Ingress got an ALB kubectl get ingress demo ADDRESS = an …elb.amazonaws.com name
Reconcile clean kubectl -n kube-system logs deploy/aws-load-balancer-controller successfully reconciled, no AccessDenied
ALB exists in AWS aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName,\k8s-default-demo`)].DNSName’` The DNS name
Targets are pod IPs aws elbv2 describe-target-health --target-group-arn <arn> healthy, IPs in the pod CIDR
It serves curl -s http://$ALB/ The echoserver response

13. Destroy. ⚠️ Order matters — this is the orphaned-ALB trap.

terraform destroy -auto-approve
# kubernetes_ingress_v1.demo: Destroying...        # ← ALB torn down FIRST (reverse dep order)
# kubernetes_ingress_v1.demo: Destruction complete after 41s
# helm_release.lbc: Destruction complete after 9s
# module.lbc_irsa...: Destruction complete after 3s
# Destroy complete! Resources: 8 destroyed.

Because kubernetes_ingress_v1.demo has depends_on = [helm_release.lbc], Terraform destroys it before the controller — so the controller is still running to delete the ALB, its target group and its managed security group. That’s the whole point of keeping the Ingress in Terraform. Confirm nothing was left behind:

aws elbv2 describe-load-balancers \
  --query 'LoadBalancers[?contains(LoadBalancerName,`k8s-`)].LoadBalancerName' --output text
# (empty)  ← no orphaned ALB

If you had created any Ingress with kubectl instead of Terraform, terraform destroy would not know about it, would tear down the controller first, and the ALB would be orphaned — still billing, with no controller left to remove it. That’s the rule to burn in: delete every Ingress (and Service type: LoadBalancer) before the controller goes.

Variables, outputs & making it reusable

The demo hard-codes one cluster’s coordinates. Turning this into a module means parameterising exactly the values a second cluster would differ on — its name, endpoint, CA, VPC, OIDC provider, and the chart version — and exposing the role ARN and controller status as outputs so a downstream stack (the Ingress/SSL lesson’s ExternalDNS, say) can consume them. The variable set is already close; a reusable module wraps iam.tf + helm.tf + ingressclass.tf (not the demo app) behind that input shape:

module "lb_controller" {
  source = "./modules/aws-lb-controller"

  cluster_name      = module.eks.cluster_name
  region            = var.region
  vpc_id            = module.vpc.vpc_id
  oidc_provider_arn = module.eks.oidc_provider_arn
  cluster_endpoint  = module.eks.cluster_endpoint
  cluster_ca_data   = module.eks.cluster_certificate_authority_data
  chart_version     = "1.11.0"
}

You rarely need to author that from scratch, though — three community options cover most needs, and knowing when to reach for each saves a day:

Approach You maintain Reach for it when
Raw helm_release + IRSA module (this lesson) The release + role wiring You want to understand it, or need custom values
terraform-aws-modules/eks blueprints / addons Just the toggle You already use that EKS module — it can install the LBC as an addon
aws-ia/eks-blueprints-addons/aws The addon inputs Installing many addons (LBC, ExternalDNS, Karpenter, metrics) together, consistently

The eks-blueprints-addons module is the production sweet spot when you’re installing a fleet of controllers: it wires each addon’s IRSA role and Helm release for you and keeps versions consistent. Roll your own (as here) when you want full control of the values, an unusual chart version, or you’re learning what the module hides. Whichever you pick, expose iam_role_arn and the release name as outputs — downstream Ingress/DNS stacks consume exactly those.

Common mistakes and troubleshooting

The controller has two signature failures, and each has a signature location. CrashLoopBackOff is almost always an IRSA/policy problem — the pod can’t get or use AWS credentials. An Ingress with a blank ADDRESS is almost always subnet tags or the IngressClass — the controller is running but can’t (or won’t) build the ALB. Start every investigation with two commands: kubectl -n kube-system get pods -l app.kubernetes.io/name=aws-load-balancer-controller and kubectl -n kube-system logs deploy/aws-load-balancer-controller. The logs name the exact cause. This is the table to keep open:

Symptom Likely cause Fix
Controller CrashLoopBackOff, logs AccessDenied / is not authorized IRSA role missing/incomplete policy, or SA :sub mismatch Attach the official policy; verify trust :sub = system:serviceaccount:kube-system:aws-load-balancer-controller
Controller CrashLoopBackOff, WebIdentityErr / no creds SA not annotated with the role ARN (escaping bug) Fix serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn; check kubectl get sa … -o yaml
Pod starts, Ingress ADDRESS blank, log unable to discover at least one subnet Subnet tags missing Tag public subnets kubernetes.io/role/elb=1 (+ …/cluster/<name>), or set the subnets annotation
Ingress ADDRESS blank, no controller log line for it Wrong/absent ingressClassName, or no IngressClass Create the alb IngressClass; set ingressClassName: alb on the Ingress
apply fails: Kubernetes cluster unreachable / connection refused Provider-after-cluster ordering (cold apply) Two-module split, or -target the cluster first; use exec auth
apply fails installing chart: failed to create: … webhook … no endpoints available Webhook not ready yet, or a partial prior install Re-apply; ensure replicaCount ≥ 1 healthy; delete a broken release and re-install
Error: Kubernetes cluster unreachable: … token is expired aws_eks_cluster_auth token expired mid-apply Switch to the exec plugin (fresh token per call)
Targets unhealthy, 502/503 at the ALB Health check path/port wrong, or SG blocks the ALB Fix healthcheck-path/port annotations; the node/pod SG must allow the ALB SG
ALB created but target-type: instance and no traffic Service isn’t NodePort (instance mode needs it) Use target-type: ip (VPC CNI), or make the Service NodePort
terraform destroy finished but an ALB still bills An Ingress was created outside Terraform (kubectl) Delete all Ingresses/LB Services before destroying the controller

Beyond the table, the traps that cost real time:

The IRSA :sub typo. The trust policy pins one exact ServiceAccount. If the chart’s SA name, its namespace, and the trust condition don’t all say kube-system / aws-load-balancer-controller, the pod assumes no role and dies with AccessDenied. Verify the actual annotation on the running SA (kubectl -n kube-system get sa aws-load-balancer-controller -o jsonpath='{.metadata.annotations}') and confirm it equals the role ARN, then confirm the role’s trust condition matches. This one check resolves the majority of CrashLoops.

The escaped-dots annotation. serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn looks absurd, and that’s the point: Helm splits on ., so the literal dots in the key must be escaped, and HCL doubles each backslash. If the annotation lands under the wrong key the SA silently has no role — same AccessDenied symptom, different root cause. Prefer creating the SA with kubernetes_service_account_v1 (a clean annotation map) if the escaping keeps biting you.

The webhook chicken-and-egg. The chart installs a validating/mutating webhook that the API server calls whenever an Ingress is created or changed. If the controller pods aren’t healthy, the webhook has no endpoints, and creating an Ingress (or even the IngressClass, in some versions) fails with no endpoints available for service "aws-load-balancer-webhook-service". It’s a readiness race, not a config error: make sure replicaCount ≥ 1 is actually Ready, and re-apply. A depends_on from the Ingress to the release (as in the demo) mostly sequences this correctly.

The subnet-tag blind spot. The controller doesn’t fail loudly here — the Ingress just sits with a blank ADDRESS, and only the controller log says unable to discover at least one subnet. Public subnets need kubernetes.io/role/elb=1 for internet-facing ALBs; private subnets need kubernetes.io/role/internal-elb=1 for internal ones. Tag them in the VPC module, or (escape hatch) list them per-Ingress with alb.ingress.kubernetes.io/subnets.

The orphaned ALB on destroy. Because the ALB, its target group and its managed SG are created by the controller and not in Terraform state, destroying the controller before its Ingresses leaves them stranded — billing forever, with nothing to reap them. Keep every Ingress and LB-typed Service as a Terraform resource with depends_on the release (so they destroy first), and delete any kubectl-created ones by hand before terraform destroy. And set region/vpcId explicitly rather than relying on IMDS auto-detect (which fails on Fargate and some node configs) — a wrong vpcId builds ALBs in a network that never routes.

Cost, cleanup & production notes

The controller itself is free — it’s a Deployment running on nodes you already pay for, plus two free IAM objects. The cost is entirely in what it creates: every Ingress (unless merged into an IngressGroup) becomes an ALB that bills hourly plus LCUs; every LB-typed Service becomes an NLB. Indicative Mumbai / ap-south-1, July 2026:

Resource Rough monthly if left up Notes
The controller (pods) ₹0 extra Runs on existing nodes
IAM role + policy Free Control-plane glue
Each ALB it creates ~₹1,400 + LCU (~$16+) One per Ingress, unless you use group.name
Each NLB it creates ~₹1,400 + NLCU (~$16+) One per LB-typed Service
This demo, one week ~₹350 (~$4) One ALB — which is why you destroy it

Two cost levers matter in production. IngressGroups (alb.ingress.kubernetes.io/group.name) merge many Ingresses onto one shared ALB, so ten services cost one ALB, not ten — the single biggest saving on a busy cluster. And destroying in order (Ingresses before the controller) is a cost control as much as a hygiene one: an orphaned ALB bills indefinitely.

Cleanup is terraform destroy, and it’s clean only if your Ingresses are Terraform-managed with depends_on the release (they destroy first, the controller reaps their ALBs, then the controller goes). Always confirm with aws elbv2 describe-load-balancers that no k8s-… load balancer survives.

Production hardening, the five that matter:

  1. Two-module layout, remote locked state. Build the cluster in one root module and the controller (and other addons) in a second that reads the cluster via remote_state/data sources. This sidesteps the provider-after-cluster trap entirely and keeps blast radius small. Use S3 + DynamoDB locking on both.
  2. Pin the chart and the policy together. The IAM policy must match the controller version — newer controllers call newer actions. Upgrade version (chart) and iam_policy.json (or the IRSA module version) in the same change, never one alone, or the controller AccessDenieds on an action the old policy lacks.
  3. Least privilege via IRSA, not the node role. Never grant load-balancer permissions to the node instance profile — scope them to the controller’s ServiceAccount through IRSA (as here). That’s the whole security upgrade over the in-tree provider; don’t undo it.
  4. HA and PodDisruptionBudget. Run replicaCount = 2 (leader-elected) so a node drain doesn’t stop reconciliation, and add a PodDisruptionBudget so cluster autoscaling can’t evict both replicas at once.
  5. Watch drift and orphans. Run terraform plan on a schedule to catch a console-edited Ingress annotation, and periodically list k8s-… ALBs to catch orphans from any out-of-band Ingress. Enable ALB access logs to S3 on production Ingresses (alb.ingress.kubernetes.io/load-balancer-attributes) for post-hoc 5xx forensics.

Cheat-sheet

The dense reference for installing and driving the controller — resources, the Helm values you set every time, the annotations you reach for, and the verify/troubleshoot commands:

Resource Purpose Must-set
aws_iam_policy (from file()) The official controller policy policy = file("iam_policy.json") (or use the IRSA module)
iam-role-for-service-accounts-eks module IRSA role + policy + OIDC trust attach_load_balancer_controller_policy, oidc_providers
helm_release (aws-load-balancer-controller) Installs the controller clusterName, region, vpcId, SA role-arn
kubernetes_ingress_class_v1 Registers the alb class spec.controller = eks.amazonaws.com/aws-load-balancer-controller
kubernetes_ingress_v1 Triggers an ALB ingress_class_name = "alb", alb.ingress.* annotations
provider "kubernetes" / "helm" Talk to the cluster host, cluster_ca_certificate, exec token
Helm value Set to
clusterName The cluster name (required)
region / vpcId Region + VPC (explicit beats auto-detect)
serviceAccount.create / .name true / aws-load-balancer-controller
serviceAccount.annotations.eks\.amazonaws\.com/role-arn The IRSA role ARN (mind the escaping)
replicaCount 2 (HA, leader-elected)
Key annotation (on the Ingress) Effect
alb.ingress.kubernetes.io/scheme internet-facing / internal
alb.ingress.kubernetes.io/target-type ip (EKS default) / instance
alb.ingress.kubernetes.io/group.name Merge onto a shared ALB (cost)
alb.ingress.kubernetes.io/certificate-arn ACM cert for HTTPS
alb.ingress.kubernetes.io/subnets Explicit subnets (bypass tag discovery)
Subnet tag Value On
kubernetes.io/role/elb 1 Public subnets (internet-facing ALB)
kubernetes.io/role/internal-elb 1 Private subnets (internal ALB)
kubernetes.io/cluster/<name> owned/shared All cluster subnets
Verify / debug with Command
Controller up kubectl -n kube-system get deploy aws-load-balancer-controller
Controller logs kubectl -n kube-system logs deploy/aws-load-balancer-controller
Ingress address kubectl get ingress demo
SA role annotation kubectl -n kube-system get sa aws-load-balancer-controller -o yaml
ALB exists aws elbv2 describe-load-balancers --query 'LoadBalancers[?contains(LoadBalancerName,\k8s-`)]'`

Interview and exam questions

1. What does the AWS Load Balancer Controller do, and how is it different from the old in-tree cloud provider? It’s a controller you run in the cluster that watches Ingress objects (class alb) and provisions ALBs, watches annotated Service type: LoadBalancer objects and provisions NLBs, and via the TargetGroupBinding CRD keeps an existing target group’s members synced to pods. The legacy in-tree provider (in kube-controller-manager) could only create a Classic ELB for a Service and had no Ingress support, no target-type IP, no ACM/WAF, and used the node instance profile instead of IRSA.

2. Why does the controller need IRSA rather than the node instance role? Because it calls AWS APIs (ELBv2, EC2, ACM) from a pod, and using the node instance profile would grant those permissions to every pod on the node. IRSA maps the specific kube-system:aws-load-balancer-controller ServiceAccount to a scoped IAM role through the cluster’s OIDC provider (sts:AssumeRoleWithWebIdentity), so only the controller can assume it — least privilege.

3. Walk through the IRSA trust policy for the controller. The role’s assume_role_policy allows sts:AssumeRoleWithWebIdentity for a Federated principal equal to the cluster’s OIDC provider ARN, with two StringEquals conditions: <oidc>:sub = system:serviceaccount:kube-system:aws-load-balancer-controller (pin the exact SA) and <oidc>:aud = sts.amazonaws.com. A typo in the :sub is the top cause of a CrashLoopBackOff with AccessDenied.

4. How do you install the controller in Terraform, and what values are mandatory? A helm_release of aws-load-balancer-controller from https://aws.github.io/eks-charts, pinned to a chart version. Mandatory-ish values: clusterName (required), region, vpcId, and the ServiceAccount wiring — serviceAccount.name matching the IRSA :sub, and serviceAccount.annotations.eks\.amazonaws\.com/role-arn set to the role ARN (with the dots escaped for Helm --set).

5. Explain the provider-after-cluster ordering problem. Terraform configures a provider before applying resources. If the kubernetes/helm provider is configured from an EKS cluster’s endpoint/CA that don’t exist yet (cold apply of cluster + controller in one module), provider config fails with Kubernetes cluster unreachable. depends_on doesn’t apply to providers. Fixes: a two-module split (cluster first, controller second), or terraform apply -target=module.eks then a full apply.

6. target-type: ip vs instance on EKS — which and why? ip registers pod IPs directly and the ALB routes straight to the pod ENI — one hop, no NodePort, works on Fargate; it relies on the VPC CNI giving pods routable VPC IPs, which EKS does by default, so ip is the standard choice. instance registers the node + a NodePort and adds a kube-proxy hop — only needed with a CNI that doesn’t give pods routable addresses.

7. You applied an Ingress and its ADDRESS stays blank. Diagnose. Read the controller log. unable to discover at least one subnet → the subnets aren’t tagged (kubernetes.io/role/elb=1 on public, …/internal-elb=1 on private); add the tags or set the subnets annotation. No log line for the Ingress at all → the ingressClassName is wrong/absent or the alb IngressClass doesn’t exist. AccessDenied in the log → the IRSA policy/trust is wrong.

8. How does the controller decide which subnets to place an ALB in? Auto-discovery by tag: kubernetes.io/role/elb=1 marks public subnets for internet-facing ALBs, kubernetes.io/role/internal-elb=1 marks private subnets for internal ones, and kubernetes.io/cluster/<name> associates subnets with the cluster. You can override per-Ingress with alb.ingress.kubernetes.io/subnets.

9. Why must you delete Ingresses before terraform destroy, and how do you make destroy safe? The ALB, its target group and its managed security group are created by the controller, out-of-band, and are not in Terraform state. If destroy removes the controller (the helm_release) while an Ingress still exists, nothing is left to delete the ALB and it’s orphaned (and billing). Make it safe by keeping every Ingress as a Terraform resource with depends_on the release, so it’s destroyed first (reverse dependency order) while the controller is still alive to reap the ALB.

10. What is a TargetGroupBinding, and when would you use it? A CRD the controller installs that binds an existing target-group ARN to a Kubernetes Service. You provision the ALB and target group in Terraform (aws_lb, aws_lb_target_group), and the controller only manages the membership — registering/deregistering pods. Use it when you want the load balancer owned by Terraform/GitOps but still want pods attached automatically.

11. (Terraform Associate 003) You change the helm_release chart version. What does plan show? An in-place update to the release — Helm upgrades the chart to the new version. It’s not a replacement of a load balancer (the controller manages those). Note: bump the attached IAM policy in the same change so the newer controller has permissions for any new actions it calls.

12. (Terraform Associate 003) Why prefer the exec auth plugin over aws_eks_cluster_auth for the kubernetes/helm providers? aws_eks_cluster_auth fetches a token once at plan/refresh time and it expires in ~15 minutes; a long apply (a slow Helm install) can fail mid-run with token is expired. The exec plugin invokes aws eks get-token on every API call, so the token is always fresh — at the cost of needing the aws CLI on the runner.

Key takeaways

TerraformawsEKSKubernetesAWS Load Balancer ControllerIRSAHelmhelm_releaseIngressALBIngressClassTargetGroupBindingOIDCIaC
Need this built for real?

Vinod is a Senior Cloud Architect (22+ yrs) — available for Azure / AWS / GCP architecture, landing zones, and migrations.

Work with me

Comments