AWS Application Load Balancer & Target Groups Hands-On: Listeners, Rules & Health Checks

Quick take: an Application Load Balancer (ALB) is a layer-7 reverse proxy that terminates HTTP/HTTPS on a listener, evaluates priority-ordered rules to pick a destination, and forwards to a target group whose health checks decide which registered targets are eligible. Get the four objects right — listener, rule, target group, health check — and everything else (stickiness, TLS, redirects, draining) is a knob on one of them. Get the security-group chain or the health-check matcher wrong and you will stare at 503 and “unhealthy” targets at 2 a.m. while the app runs perfectly.

A retail team shipped a routine deploy on a Friday and the site started throwing 503 Service Unavailable — intermittently at first, then constantly. The instances were up, the app booted fine, curl localhost/health returned 200 on every box. Yet the ALB insisted it had no healthy targets. The on-call engineer restarted instances (no change), doubled the Auto Scaling group (no change), and finally opened a support case. The actual cause took ninety seconds to find once someone looked in the right place: the deploy had moved the health endpoint from /health to /healthz, but the target group’s health check still probed /health, which now returned 404. The default matcher only accepts 200, so every target failed its check, the target group emptied, and the ALB — correctly — returned 503 because it had nowhere to send traffic. Not a code bug. A health-check configuration bug, three layers down from the symptom.

This article is the hands-on, production-grade guide to ALB and target groups: the whole object model, every listener and rule option, every target-group attribute, health checks and their reason codes, stickiness, cross-zone, deregistration delay and slow start, HTTPS with ACM and SNI, access logs, and the security-group pattern that trips up nearly everyone. You will build the real thing — an ALB with an HTTPS listener, two path-routed target groups behind an Auto Scaling group, health checks and a rolling target replacement — with both the aws elbv2 CLI and Terraform, then tear it down. Because this is a reference you will return to mid-incident, the options, limits, error codes and the playbook are all laid out as scannable tables. Read the prose once; keep the tables open when the pager fires.

By the end you will stop guessing. You will localise an ALB failure to the exact hop — listener, rule, target group, health check, or the security group between the ALB and its targets — and fix it because you understand what each object is for, not by restarting instances and hoping.

What problem this solves

A single server is a single point of failure and a single bottleneck. The moment you run more than one instance of anything — for availability, for scale, for zero-downtime deploys — you need something in front that spreads traffic, hides individual instances behind one stable endpoint, stops sending requests to a box that is broken, and lets you swap instances without dropping connections. That “something” is a load balancer, and on AWS the layer-7 version is the Application Load Balancer.

What breaks without it, or with it misconfigured: you deploy a new version and users get connection resets because old instances were killed mid-request (no deregistration delay). One instance wedges and keeps taking traffic because nothing checks its health (no health check, or the wrong one). You terminate TLS in ten places and rotate certs by hand (no central ACM integration). You cannot route /api to one fleet and the web UI to another without a second load balancer (no listener rules). A cache-warming instance gets hammered the instant it joins and falls over (no slow start). And the single most common failure — targets that are “unhealthy” though the application is fine — comes from a security group that does not allow the ALB to reach the target port, or a health-check matcher that does not accept what the app returns.

Who hits this: essentially every team running a web app, an API, or containers on AWS. It bites hardest on teams that treat the ALB as a black box — they create it in the console, it “works,” and then they have no mental model when it returns 503, serves the wrong certificate, or drops connections on every deploy. The fix is almost never “restart” or “scale up.” It is “find the object that is lying — the rule, the health check, or the SG chain — and make it tell the truth.”

To frame the field, here is the ALB object model — the six objects you will configure, in the order a request traverses them. Internalise this hierarchy and every setting later has an obvious home.

Learning objectives

By the end of this article you can:

• Choose correctly between ALB, NLB, Gateway Load Balancer and the legacy Classic Load Balancer using a layer-and-feature chooser, and explain why forcing one to do another’s job is the most expensive mistake in this area.
• Configure ALB listeners on HTTP and HTTPS, terminate TLS with an ACM certificate, attach multiple certs via SNI, pick a TLS security policy, and redirect HTTP→HTTPS without creating a loop.
• Write priority-ordered listener rules using every condition type (host-header, path-pattern, http-header, http-request-method, query-string, source-IP) and every action type (forward, redirect, fixed-response, authenticate-oidc, authenticate-cognito).
• Create target groups of every target type (instance, IP, Lambda), pick the protocol version (HTTP1/HTTP2/gRPC), and tune every attribute: deregistration delay, slow start, stickiness, load-balancing algorithm and cross-zone.
• Configure health checks — path, port, protocol, matcher codes, interval, timeout, healthy/unhealthy thresholds — and read every health-check reason code to diagnose an unhealthy target in seconds.
• Wire the security-group chain correctly (target SG references the ALB SG), enable access logs to S3, and set desync mitigation and drop-invalid-header defenses.
• Build the whole thing in a hands-on lab with aws elbv2 and Terraform, verify path-based routing, perform a rolling target replacement, and tear it down without leaving billable resources.
• Run a symptom → confirm → fix playbook for unhealthy targets, 503 no-healthy-targets, 502/504, redirect loops, cert/SNI mismatch, sticky imbalance and deregistration cutting connections.

Prerequisites & where this fits

You should already understand VPC basics — that a load balancer needs at least two subnets in two Availability Zones, that public/internet-facing ALBs live in public subnets while their targets usually sit in private subnets, and how security groups work as stateful allow-lists. You should be comfortable running the aws CLI (v2) with a configured profile, reading JSON output, and applying a small Terraform config. Familiarity with HTTP status codes, TLS termination and Auto Scaling groups helps but is not required.

This sits in the Networking track and is the practical companion to three articles you should keep nearby. For the decision of which front door to use, read AWS Load Balancers and API Gateway: ALB, NLB and API Gateway Compared — this article goes deep on the ALB that guide tells you to pick. For where the ALB sits in a full stack, see The Classic AWS Three-Tier Web Application Architecture, which places the ALB between the VPC edge and the Auto Scaling tier. For the DNS layer that resolves your ALB’s name and can fail traffic over between regions, see Amazon Route 53 in Practice: Records, Alias & Routing Policies. When the ALB is throwing 5xx specifically, the deep incident guide is the ALB 502/503/504 troubleshooting playbook.

Before the deep dive, here is the ELB family so you know what ALB is not, and when to reach for a sibling instead. Pick the OSI layer first; the product follows.

Core concepts

An ALB is a managed reverse proxy that AWS runs for you across the AZs you choose. It publishes a stable DNS name; behind that name AWS places elastic network interfaces (ENIs) — at least one per enabled AZ — and scales the underlying capacity up and down invisibly. A client resolves the DNS name to those IPs, opens a connection to a listener, and the ALB terminates the connection (and TLS, for HTTPS). It then makes a new connection to a chosen target. That “two connections, not one” fact explains a lot: the target sees the ALB’s IP as the source (real client IP arrives in X-Forwarded-For), the ALB can pool and reuse backend connections, and the ALB — not your app — decides retries, timeouts and health.

The request path inside the ALB is deterministic and worth memorising: listener → rules (in priority order) → matched action → target group → load-balancing algorithm picks a healthy target → forward. Every request runs this gauntlet. When something misbehaves, you localise it to one stage.

Here is the object model in one dense reference — the settings, defaults and the limit or gotcha that bites on each.

Two more mental anchors before we go option-by-option. First, the ALB is HTTP-only — it speaks HTTP/1.1, HTTP/2 and gRPC and nothing else; if you need raw TCP/UDP you want an NLB. Second, an ALB and an NLB differ on a handful of axes that decide most designs:

Listeners: protocols, ports and TLS

A listener is the front door: a protocol and a port the ALB accepts, plus a default action and — for HTTPS — TLS configuration. An ALB almost always has two listeners: :80 (HTTP) whose only job is to redirect to HTTPS, and :443 (HTTPS) that terminates TLS and does the real routing.

Protocols and ports

ALB listeners accept only HTTP and HTTPS, on any port 1–65535. Everything L7-specific — rules, WAF, X-Forwarded-*, sticky cookies — is available on both, but TLS termination, SNI, mTLS and the authenticate actions require HTTPS.

Listener and load-balancer attributes

Several defenses and behaviours live on the listener or the load balancer rather than the target group. Know these — three of them (desync_mitigation_mode, drop_invalid_header_fields, idle_timeout) show up in real incidents.

TLS termination, ACM and security policies

For an HTTPS listener you attach a certificate (ideally free and auto-renewing from AWS Certificate Manager, ACM) and choose a security policy — the negotiated set of TLS protocol versions and cipher suites. The default is modern (TLS 1.2 + 1.3); tighten it if you must, but do not loosen it without a reason.

Create the HTTPS listener with the CLI, pointing at an ACM cert ARN and the default target group:

aws elbv2 create-listener \
  --load-balancer-arn "$ALB_ARN" \
  --protocol HTTPS --port 443 \
  --ssl-policy ELBSecurityPolicy-TLS13-1-2-2021-06 \
  --certificates CertificateArn="$ACM_CERT_ARN" \
  --default-actions Type=forward,TargetGroupArn="$TG_WEB_ARN"

The Terraform equivalent, with the HTTP→HTTPS redirect listener alongside it:

resource "aws_lb_listener" "https" {
  load_balancer_arn = aws_lb.web.arn
  port              = 443
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-TLS13-1-2-2021-06"
  certificate_arn   = aws_acm_certificate.web.arn

  default_action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.web.arn
  }
}

resource "aws_lb_listener" "http_redirect" {
  load_balancer_arn = aws_lb.web.arn
  port              = 80
  protocol          = "HTTP"

  default_action {
    type = "redirect"
    redirect {
      protocol    = "HTTPS"
      port        = "443"
      status_code = "HTTP_301"
    }
  }
}

SNI: many certificates on one listener

A single HTTPS listener can serve many hostnames. You attach one default certificate plus additional certificates; the ALB uses Server Name Indication (SNI) to pick the right cert from the client’s TLS handshake. Clients that send no SNI (rare, old) get the default cert.

# Add a second hostname's cert to an existing HTTPS listener via SNI
aws elbv2 add-listener-certificates \
  --listener-arn "$HTTPS_LISTENER_ARN" \
  --certificates CertificateArn="$ACM_CERT_API_ARN"

Listener rules: conditions, actions and priority

Rules are where ALB earns “application” in its name. Each HTTPS/HTTP listener has an ordered list of rules plus a mandatory default rule (priority default, always last). For each request the ALB evaluates rules from the lowest priority number up; the first rule whose conditions all match wins, its actions run, and evaluation stops. If nothing matches, the default action runs.

Priority is everything

Priority is a positive integer 1–50000; lower numbers are evaluated first. The classic bug is a broad rule shadowing a specific one — a path-pattern /* at priority 5 will swallow /api/* at priority 10, and /api traffic never reaches the API fleet. Order specific-before-general.

Conditions: the “if”

A rule can combine up to five condition types; all must match (logical AND). Within one condition you can list multiple values (logical OR).

Actions: the “then”

An action is what a matched rule does. Some actions are terminal (forward, redirect, fixed-response); the authenticate actions run first and then fall through to a forward.

A weighted forward lists up to five target groups with integer weights and optional target-group stickiness, which is how you do canary and blue/green at the ALB — shift 5% then 50% then 100% by editing weights, no DNS change.

Create the /api/* rule with the CLI:

aws elbv2 create-rule \
  --listener-arn "$HTTPS_LISTENER_ARN" \
  --priority 10 \
  --conditions Field=path-pattern,Values='/api/*' \
  --actions Type=forward,TargetGroupArn="$TG_API_ARN"

The Terraform rule, and a maintenance-mode fixed-response example:

resource "aws_lb_listener_rule" "api" {
  listener_arn = aws_lb_listener.https.arn
  priority     = 10

  condition {
    path_pattern { values = ["/api/*"] }
  }
  action {
    type             = "forward"
    target_group_arn = aws_lb_target_group.api.arn
  }
}

resource "aws_lb_listener_rule" "maintenance" {
  listener_arn = aws_lb_listener.https.arn
  priority     = 5
  condition {
    path_pattern { values = ["/admin/*"] }
  }
  condition {
    source_ip { values = ["203.0.113.0/24"] }   # office egress only
  }
  action {
    type = "fixed-response"
    fixed_response {
      content_type = "text/plain"
      message_body = "Maintenance in progress"
      status_code  = "503"
    }
  }
}

Rule and load-balancer limits

Enumerate the real quotas so you design within them (several are soft and raisable via Service Quotas).

Path-pattern matching: the details that bite

Path patterns are case-sensitive, match only the path (not the query string), and support * (any sequence) and ? (any single char). A few worked examples:

Target groups: types, protocols and registration

A target group is a named pool of destinations plus the rules for health-checking and load-balancing them. Listener actions point at target groups; targets register into them. One target group can serve many rules, and one target can belong to many target groups.

Target types

The target type is fixed at creation and decides what you register and how the ALB reaches it.

Protocol version: HTTP1, HTTP2, gRPC

The target-group protocol version governs how the ALB talks to targets. Pick GRPC for gRPC services (health checks use gRPC status), HTTP2 for HTTP/2 cleartext backends, else HTTP1.

Target-group attributes — the tuning surface

This is where deregistration delay, slow start, stickiness, the balancing algorithm and cross-zone live. Learn every row; three of them are in every ALB incident.

Load-balancing algorithm

The algorithm decides which healthy target gets the next request. round_robin is even and simple; least_outstanding_requests sends to the target with the fewest in-flight requests (better when request cost varies wildly); weighted_random supports anomaly mitigation.

Registration lifecycle and target health states

A target moves through states from the moment it registers. Reading the state (and its reason code) is the fastest ALB diagnosis you have.

Create a target group and register instances with the CLI:

aws elbv2 create-target-group \
  --name tg-web --protocol HTTP --protocol-version HTTP1 \
  --port 80 --vpc-id "$VPC_ID" --target-type instance \
  --health-check-protocol HTTP --health-check-path /healthz \
  --matcher HttpCode=200 \
  --health-check-interval-seconds 15 --health-check-timeout-seconds 5 \
  --healthy-threshold-count 3 --unhealthy-threshold-count 2

aws elbv2 modify-target-group-attributes \
  --target-group-arn "$TG_WEB_ARN" \
  --attributes Key=deregistration_delay.timeout_seconds,Value=120 \
               Key=slow_start.duration_seconds,Value=60

Health checks: path, matcher and thresholds

The health check is the ALB’s opinion of whether a target can take traffic. Get it wrong and you either send traffic to broken boxes (checks too lenient) or empty the target group and return 503 (checks too strict, wrong path, or wrong matcher). This one object causes more ALB pages than any other.

Every health-check setting

Detection-time math

The time to evict a bad target is roughly UnhealthyThreshold × Interval; to restore it, HealthyThreshold × Interval. Defaults (2 × 30 = 60s to evict; 5 × 30 = 150s to restore) are conservative. For fast blue/green, tighten to interval 10, unhealthy 2, healthy 2 → 20s each way. Do not go so aggressive that a single slow GC pause flaps a target.

Health-check reason codes — the diagnosis table

When a target is unhealthy, describe-target-health returns a reason code that names the cause. This table turns a vague “unhealthy” into a specific fix.

Check target health from the CLI — the single most useful ALB command:

aws elbv2 describe-target-health --target-group-arn "$TG_WEB_ARN" \
  --query 'TargetHealthDescriptions[].{Id:Target.Id,State:TargetHealth.State,Reason:TargetHealth.Reason,Desc:TargetHealth.Description}' \
  --output table

Expected output when the SG chain is broken (the classic case):

------------------------------------------------------------------
|                     DescribeTargetHealth                       |
+--------------+-----------+----------------+--------------------+
|      Id      |   State   |     Reason     |        Desc        |
+--------------+-----------+----------------+--------------------+
| i-0abc123... | unhealthy | Target.Timeout | Health checks failed|
+--------------+-----------+----------------+--------------------+

Stickiness, cross-zone, draining and slow start

Four target-group behaviours deserve their own treatment because each has a subtle failure mode.

Stickiness

Stickiness pins a client’s requests to one target — needed for apps that keep server-side session state. ALB supports two flavours.

Stickiness is the usual cause of AZ imbalance: a long sticky duration pins established users to their first target, so a scale-out adds cold targets that get little traffic while old ones stay hot. Shorten the duration or move session state to Redis/DynamoDB and turn stickiness off.

Cross-zone load balancing

Cross-zone decides whether a node in AZ-a can send to targets in AZ-b. On the ALB it is on by default and free; on the NLB it is off by default and enabling it incurs inter-AZ data transfer charges. Without cross-zone, uneven target counts per AZ cause uneven load.

Deregistration delay (connection draining)

When a target deregisters (deploy, scale-in), the ALB moves it to draining: it stops sending new requests but lets in-flight ones finish for deregistration_delay.timeout_seconds (default 300). Set this ≥ your longest legitimate request; too short and you cut long uploads/downloads mid-flight, too long and deploys crawl.

Slow start

Slow start ramps traffic to a newly healthy target linearly over slow_start.duration_seconds (30–900, default off) instead of hitting it at full share instantly. Use it for apps with cold caches or JIT warm-up; a cold target thrown to full load can time out and flap. Slow start is incompatible with the least_outstanding_requests algorithm on the ramp.

Here is the full set of timers so you can reason about a deploy end to end:

Access logs, security groups and desync mitigation

Three operational concerns finish the ALB picture: how you get request-level logs, the security-group pattern that makes targets reachable, and the request-smuggling defenses.

Access logs to S3

ALB access logs capture every request as a line in a gzip file delivered to S3 every ~5 minutes. They are the ground truth for “what did the ALB actually do” — the client IP, the chosen target, the ALB status vs the target status, and the processing times that separate an ALB problem from an app problem. They are off by default; enable them and grant the ELB log-delivery account (or the regional log-delivery service principal) s3:PutObject on the bucket.

The security-group chain

This is the pattern people get wrong, and it produces the “unhealthy though the app is fine” page. There are two security groups and the target’s SG must reference the ALB’s SG as its source, not a CIDR.

The health check rides the same path as real traffic, so if the target SG does not allow the ALB SG on the (health) port, health checks time out (Target.Timeout) and the target is evicted — even though the app is perfectly healthy. Reference the SG id, never a CIDR, so it keeps working as the ALB scales and changes IPs.

# Target SG allows the ALB SG on port 80 (traffic + health)
aws ec2 authorize-security-group-ingress \
  --group-id "$TARGET_SG" --protocol tcp --port 80 \
  --source-group "$ALB_SG"

resource "aws_security_group_rule" "target_from_alb" {
  type                     = "ingress"
  security_group_id        = aws_security_group.target.id
  from_port                = 80
  to_port                  = 80
  protocol                 = "tcp"
  source_security_group_id = aws_security_group.alb.id
}

Desync mitigation and header defenses

HTTP request smuggling / desync attacks exploit disagreements about where one request ends and the next begins. ALB’s desync_mitigation_mode classifies requests and can block ambiguous ones.

Pair it with drop_invalid_header_fields.enabled=true to strip malformed headers before they reach the app.

Architecture at a glance

The diagram traces one HTTPS request through the exact system you build in the lab. Read it left to right. A client opens HTTPS :443 to the internet-facing ALB, whose security group admits :443 from the internet and whose :80 listener 301-redirects to :443. The :443 listener terminates TLS with an ACM certificate (additional hostnames attach via SNI) and evaluates its rules in priority order: priority 10, path-pattern /api/*, forwards to target group A; everything else falls through to the default action → target group B. Each target group runs its own health check and forwards to EC2 instances that an Auto Scaling group keeps spread across two AZ subnets inside the VPC, with cross-zone balancing on so both AZs share load evenly. The numbered badges sit on the exact hop where each classic failure bites — the SG chain, the TLS/redirect, rule priority, the health check and 503, deregistration/slow-start during a rolling replace, and cross-zone/stickiness imbalance.

Real-world scenario

Zeplo Retail runs an online store on AWS: a server-rendered web frontend and a separate JSON API, both on EC2 behind one ALB. The web tier is tg-web (default action); the API is tg-api (path-pattern /api/*, priority 10). Both fleets live in an Auto Scaling group across two AZs, targets in private subnets, ALB in public subnets. Traffic is steady at ~1,200 requests/second with predictable Black-Friday spikes.

The incident started with a “harmless” refactor. The API team renamed their health endpoint from /api/health to /api/_healthz and shipped it through an instance refresh on the ASG. Within two minutes, /api/* requests began returning 503. The web store stayed up — only the API was down — which is itself a clue: a total outage points at the ALB or DNS, a per-path outage points at one target group. The on-call engineer ran the one command that matters:

aws elbv2 describe-target-health --target-group-arn "$TG_API_ARN" \
  --query 'TargetHealthDescriptions[].TargetHealth' --output table

Every API target read unhealthy with reason Target.ResponseCodeMismatch. The health check still probed /api/health, which now returned 404, and the matcher only accepted 200. As the instance refresh rolled, each new target failed its check, never entered rotation, and the old ones were already terminated — so tg-api emptied and the ALB returned 503 on /api/*. CloudWatch confirmed it: HealthyHostCount for tg-api had fallen to 0 while tg-web stayed flat.

The fix was one command — repoint the health check — and the target group refilled within one healthy-threshold cycle:

aws elbv2 modify-target-group \
  --target-group-arn "$TG_API_ARN" --health-check-path /api/_healthz

The retro produced three durable changes. First, the health-check path became a variable wired to the same constant the app uses, in both the Terraform and the app config, so they can never drift again. Second, they set the ASG instance refresh to a healthy-percentage minimum of 90% so a batch of failing new targets pauses the rollout instead of draining the old ones first. Third, they added a CloudWatch alarm on HealthyHostCount < 2 per target group wired to the pager — so next time the alert fires before customers see 503, not after. Total customer-facing impact the day of the incident: eleven minutes. With the alarm, the same root cause would now surface in under two.

Advantages and disadvantages

The advantages matter most for standard web and API workloads — anything HTTP where you want content-based routing, TLS offload and zero-downtime deploys. The disadvantages bite when you have non-HTTP protocols (reach for an NLB), need a hard static IP for a partner allow-list (front the ALB with an NLB, or use Global Accelerator), or run extreme connection churn where LCU costs climb — measure with the LCU dimensions in Cost & sizing before assuming.

Hands-on lab

You will build a working ALB with an HTTPS listener, two path-routed target groups behind an Auto Scaling group, verify routing, perform a rolling target replacement, then tear everything down. Budget ~30 minutes.

⚠️ Cost warning. An ALB bills per hour (~$0.0225/hr ≈ ₹1.9/hr, ~₹1,360/month) plus LCU-hours even at near-zero traffic, and the two t3.small instances bill per second. This is not entirely free-tier. Do the whole lab in one sitting and run the teardown at the end. Total cost for a 30-minute run is well under ₹20, but a forgotten ALB is ~₹1,360/month.

Prerequisites

• AWS CLI v2 configured (aws sts get-caller-identity works), a default VPC with two public subnets in two AZs, and permission for elasticloadbalancing:*, ec2:*, autoscaling:*, acm:*.
• An ACM certificate for a domain you control (or, for a throwaway lab, import a self-signed cert — the ALB will still serve it; browsers will warn). Set ACM_CERT_ARN to its ARN.

Step 1 — Environment variables

export AWS_REGION=ap-south-1
VPC_ID=$(aws ec2 describe-vpcs --filters Name=isDefault,Values=true \
  --query 'Vpcs[0].VpcId' --output text)
SUBNETS=$(aws ec2 describe-subnets --filters Name=vpc-id,Values=$VPC_ID \
  --query 'Subnets[0:2].SubnetId' --output text)
SUBNET_A=$(echo $SUBNETS | cut -d' ' -f1); SUBNET_B=$(echo $SUBNETS | cut -d' ' -f2)
AMI=$(aws ssm get-parameters --names \
  /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
  --query 'Parameters[0].Value' --output text)
echo "VPC=$VPC_ID  A=$SUBNET_A  B=$SUBNET_B  AMI=$AMI"

Step 2 — Security groups (the chain)

ALB_SG=$(aws ec2 create-security-group --group-name lab-alb-sg \
  --description "ALB SG" --vpc-id $VPC_ID --query GroupId --output text)
TGT_SG=$(aws ec2 create-security-group --group-name lab-tgt-sg \
  --description "Target SG" --vpc-id $VPC_ID --query GroupId --output text)

# ALB accepts 80/443 from the internet
aws ec2 authorize-security-group-ingress --group-id $ALB_SG \
  --protocol tcp --port 443 --cidr 0.0.0.0/0
aws ec2 authorize-security-group-ingress --group-id $ALB_SG \
  --protocol tcp --port 80 --cidr 0.0.0.0/0
# Targets accept 80 ONLY from the ALB SG (traffic + health checks)
aws ec2 authorize-security-group-ingress --group-id $TGT_SG \
  --protocol tcp --port 80 --source-group $ALB_SG

Step 3 — The ALB

ALB_ARN=$(aws elbv2 create-load-balancer --name lab-alb \
  --type application --scheme internet-facing \
  --subnets $SUBNET_A $SUBNET_B --security-groups $ALB_SG \
  --query 'LoadBalancers[0].LoadBalancerArn' --output text)
ALB_DNS=$(aws elbv2 describe-load-balancers --load-balancer-arns $ALB_ARN \
  --query 'LoadBalancers[0].DNSName' --output text)
echo "ALB DNS: $ALB_DNS"

Step 4 — Two target groups with health checks

TG_WEB_ARN=$(aws elbv2 create-target-group --name lab-tg-web \
  --protocol HTTP --port 80 --vpc-id $VPC_ID --target-type instance \
  --health-check-path / --matcher HttpCode=200 \
  --health-check-interval-seconds 15 --healthy-threshold-count 2 \
  --query 'TargetGroups[0].TargetGroupArn' --output text)

TG_API_ARN=$(aws elbv2 create-target-group --name lab-tg-api \
  --protocol HTTP --port 80 --vpc-id $VPC_ID --target-type instance \
  --health-check-path /api/health --matcher HttpCode=200 \
  --health-check-interval-seconds 15 --healthy-threshold-count 2 \
  --query 'TargetGroups[0].TargetGroupArn' --output text)

# Tune drain + slow start on the web TG
aws elbv2 modify-target-group-attributes --target-group-arn $TG_WEB_ARN \
  --attributes Key=deregistration_delay.timeout_seconds,Value=60 \
               Key=slow_start.duration_seconds,Value=30

Step 5 — Launch template + Auto Scaling group

The user-data serves / and /api/health so both target groups pass:

USERDATA=$(printf '#!/bin/bash\ndnf install -y httpd\nmkdir -p /var/www/html/api\necho "web $(hostname)" > /var/www/html/index.html\necho "ok" > /var/www/html/api/health\nsystemctl enable --now httpd\n' | base64)

LT_ID=$(aws ec2 create-launch-template --launch-template-name lab-lt \
  --launch-template-data "{\"ImageId\":\"$AMI\",\"InstanceType\":\"t3.small\",\"SecurityGroupIds\":[\"$TGT_SG\"],\"UserData\":\"$USERDATA\"}" \
  --query 'LaunchTemplate.LaunchTemplateId' --output text)

aws autoscaling create-auto-scaling-group --auto-scaling-group-name lab-asg \
  --launch-template LaunchTemplateId=$LT_ID --min-size 2 --max-size 4 \
  --desired-capacity 2 --vpc-zone-identifier "$SUBNET_A,$SUBNET_B" \
  --target-group-arns $TG_WEB_ARN $TG_API_ARN \
  --health-check-type ELB --health-check-grace-period 60

Step 6 — Listeners: HTTPS + HTTP→HTTPS redirect

HTTPS_LISTENER=$(aws elbv2 create-listener --load-balancer-arn $ALB_ARN \
  --protocol HTTPS --port 443 \
  --ssl-policy ELBSecurityPolicy-TLS13-1-2-2021-06 \
  --certificates CertificateArn=$ACM_CERT_ARN \
  --default-actions Type=forward,TargetGroupArn=$TG_WEB_ARN \
  --query 'Listeners[0].ListenerArn' --output text)

aws elbv2 create-listener --load-balancer-arn $ALB_ARN \
  --protocol HTTP --port 80 \
  --default-actions 'Type=redirect,RedirectConfig={Protocol=HTTPS,Port=443,StatusCode=HTTP_301}'

Step 7 — The /api/* rule

aws elbv2 create-rule --listener-arn $HTTPS_LISTENER --priority 10 \
  --conditions Field=path-pattern,Values='/api/*' \
  --actions Type=forward,TargetGroupArn=$TG_API_ARN

Step 8 — Verify routing and health

Wait ~90 seconds for instances and health checks, then:

# Both target groups should show healthy targets
aws elbv2 describe-target-health --target-group-arn $TG_WEB_ARN \
  --query 'TargetHealthDescriptions[].TargetHealth.State' --output text
# Expected: healthy healthy

# Default path → web fleet; /api/* → api fleet (self-signed = -k)
curl -sk https://$ALB_DNS/         # -> "web ip-10-...."
curl -sk https://$ALB_DNS/api/health   # -> "ok"
# HTTP redirects to HTTPS (301)
curl -skI http://$ALB_DNS/ | head -1   # -> HTTP/1.1 301 Moved Permanently

Step 9 — Rolling target replacement

Trigger an ASG instance refresh and watch targets drain and re-register — a zero-downtime rolling replacement:

aws autoscaling start-instance-refresh --auto-scaling-group-name lab-asg \
  --preferences MinHealthyPercentage=50,InstanceWarmup=60

# Watch states cycle: healthy -> draining (old) ; initial -> healthy (new)
watch -n5 "aws elbv2 describe-target-health --target-group-arn $TG_WEB_ARN \
  --query 'TargetHealthDescriptions[].TargetHealth.State' --output text"

You will see an old target go draining (finishing in-flight requests for the 60s deregistration delay) while a new one goes initial → healthy and ramps over the 30s slow start. Throughout, curl -sk https://$ALB_DNS/ keeps returning 200 — that is the whole point.

Step 10 — Lab resource summary and teardown

aws autoscaling delete-auto-scaling-group --auto-scaling-group-name lab-asg --force-delete
aws elbv2 delete-load-balancer --load-balancer-arn $ALB_ARN
sleep 30   # let ENIs detach before deleting TGs/SGs
aws elbv2 delete-target-group --target-group-arn $TG_WEB_ARN
aws elbv2 delete-target-group --target-group-arn $TG_API_ARN
aws ec2 delete-launch-template --launch-template-id $LT_ID
aws ec2 delete-security-group --group-id $TGT_SG
aws ec2 delete-security-group --group-id $ALB_SG

Terraform equivalent

The whole lab as HCL (assumes vpc_id, subnet_ids, acm_cert_arn, ami variables). Apply with terraform init && terraform apply, destroy with terraform destroy (⚠️ same ALB-hour cost while it exists):

resource "aws_security_group" "alb" {
  name   = "lab-alb-sg"
  vpc_id = var.vpc_id
  ingress { from_port = 443, to_port = 443, protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] }
  ingress { from_port = 80,  to_port = 80,  protocol = "tcp", cidr_blocks = ["0.0.0.0/0"] }
  egress  { from_port = 0,   to_port = 0,   protocol = "-1",  cidr_blocks = ["0.0.0.0/0"] }
}

resource "aws_security_group" "target" {
  name   = "lab-tgt-sg"
  vpc_id = var.vpc_id
  ingress {
    from_port       = 80
    to_port         = 80
    protocol        = "tcp"
    security_groups = [aws_security_group.alb.id]   # reference the ALB SG, not a CIDR
  }
  egress { from_port = 0, to_port = 0, protocol = "-1", cidr_blocks = ["0.0.0.0/0"] }
}

resource "aws_lb" "web" {
  name               = "lab-alb"
  load_balancer_type = "application"
  internal           = false
  security_groups    = [aws_security_group.alb.id]
  subnets            = var.subnet_ids
}

resource "aws_lb_target_group" "web" {
  name       = "lab-tg-web"
  port       = 80
  protocol   = "HTTP"
  vpc_id     = var.vpc_id
  target_type = "instance"
  deregistration_delay = 60
  slow_start           = 30
  health_check {
    path                = "/"
    matcher             = "200"
    interval            = 15
    healthy_threshold   = 2
    unhealthy_threshold = 2
  }
}

resource "aws_lb_target_group" "api" {
  name        = "lab-tg-api"
  port        = 80
  protocol    = "HTTP"
  vpc_id      = var.vpc_id
  target_type = "instance"
  health_check { path = "/api/health", matcher = "200", interval = 15, healthy_threshold = 2 }
}

resource "aws_lb_listener" "https" {
  load_balancer_arn = aws_lb.web.arn
  port              = 443
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-TLS13-1-2-2021-06"
  certificate_arn   = var.acm_cert_arn
  default_action { type = "forward", target_group_arn = aws_lb_target_group.web.arn }
}

resource "aws_lb_listener" "http" {
  load_balancer_arn = aws_lb.web.arn
  port              = 80
  protocol          = "HTTP"
  default_action {
    type = "redirect"
    redirect { protocol = "HTTPS", port = "443", status_code = "HTTP_301" }
  }
}

resource "aws_lb_listener_rule" "api" {
  listener_arn = aws_lb_listener.https.arn
  priority     = 10
  condition { path_pattern { values = ["/api/*"] } }
  action { type = "forward", target_group_arn = aws_lb_target_group.api.arn }
}

Common mistakes & troubleshooting

This is the section you will return to. Start with the playbook — symptom, root cause, exact confirm, fix — then the two reference tables and the decision table.

ALB-generated HTTP status codes

These are codes the ALB itself returns (elb_status_code in the access log). When target_status_code is blank or differs, the ALB — not your app — produced the response.

Decision table — from symptom to first move

The three nastiest, explained

Unhealthy-but-fine targets (the SG chain). The health check uses the same network path as real traffic and originates from the ALB’s ENIs. If the target security group’s ingress does not list the ALB SG as source on the target/health port, checks time out (Target.Timeout) and every target is evicted — producing 503 while curl localhost on each box returns 200. Always reference the ALB SG id, never a CIDR, so it survives ALB IP changes. This one failure accounts for the majority of “ALB is broken” tickets.

Redirect loops behind the ALB. You terminate TLS at the ALB and forward plain HTTP to the target. The app, seeing an HTTP request, “helpfully” redirects to HTTPS — which comes back through the ALB as HTTP again, forever. The ALB tells the app the truth in X-Forwarded-Proto: https; the app must only redirect when that header is not https. Frameworks (Rails force_ssl, Django SECURE_PROXY_SSL_HEADER, nginx) all have a “trust the forwarded proto” switch — set it.

Deregistration cutting long requests. A deploy or scale-in deregisters a target; the ALB stops new requests but only drains in-flight ones for deregistration_delay (default 300s). If a client is mid-way through a 6-minute report export and the delay is 120s, the ALB force-closes at 120s and the client sees a reset. Set the delay to at least your longest legitimate request, and prefer graceful in-app shutdown (stop accepting, finish in-flight, exit) alongside it.

Best practices

• Two listeners, always: :80 that does nothing but 301→:443, and :443 that terminates TLS and routes. Never serve real traffic on plain HTTP.
• Reference the ALB SG by id in the target SG’s ingress — never a CIDR. It survives scaling and is the least-privilege source.
• Point health checks at a cheap, dedicated endpoint (/healthz) that returns fast and does not hammer the database on every probe — but does reflect real readiness.
• Wire the health-check path to the same constant the app uses, in IaC and app config, so a rename can’t silently break the check (the Zeplo incident).
• Set deregistration delay ≥ your longest request and enable slow start for apps with cold caches, so deploys don’t cut connections or overwhelm cold targets.
• Keep the app’s keep-alive timeout above the ALB idle timeout (60s default) to avoid 502s from races where the target closes a connection the ALB reuses.
• Order rules specific-before-general; give the catch-all the highest priority number. Test both /api and /api/x.
• Use ACM certs (free, auto-renewing) and the modern default TLS policy; only tighten, never loosen. Attach extra hostnames via SNI.
• Enable access logs to S3 and set CloudWatch alarms on HealthyHostCount < N, HTTPCode_ELB_5XX_Count and TargetResponseTime — alert before customers do.
• Turn on deletion protection in production and manage the ALB in IaC so changes are reviewed.
• Right-size health-check aggressiveness: interval 10–15s, thresholds 2–3 — fast enough to react, slow enough not to flap on a GC pause.
• Use weighted target groups for canary/blue-green instead of DNS changes — instant, reversible traffic shifts.

Security notes

The ALB is your security boundary between the internet and your compute, so treat it as one.

Least privilege is the theme: the internet reaches only the ALB’s listener ports, the ALB reaches only the target port, and the target reaches only its own dependencies. Put WAF in front for anything public, and terminate TLS at the ALB so certificate management is centralized and auditable.

Cost & sizing

ALB pricing has two parts: a flat per-hour charge for the load balancer, plus Load Balancer Capacity Units (LCU-hours). You pay for the higher of the flat-hour baseline or your LCU consumption — so at low traffic the hourly charge dominates, and at scale the LCUs do. An LCU is the maximum across four dimensions, each normalized so one LCU equals whichever dimension you hit first.

Rough figures (us-east-1 / ap-south-1 are close): ALB ~$0.0225/ALB-hour (≈ ₹1.9/hr, ~₹1,360/month) plus ~$0.008/LCU-hour. A small app doing ~1 GB/hr and a few hundred connections/sec sits near 1–3 LCUs — call it ~₹1,360 (hourly) + ~₹1,500 (LCU) ≈ ₹2,900/month. There is a limited free tier (750 ALB-hours + 15 LCUs/month for 12 months on new accounts), but a production ALB is not free.

Sizing is mostly about not sending avoidable bytes through the ALB (front static content with CloudFront/S3), consolidating rules (you get 10 free rule-evals per request), and not leaving idle ALBs running — the single biggest waste in this service.

Interview & exam questions

Q1. A target is unhealthy but curl localhost/health on the box returns 200. What’s the most likely cause? The target’s security group doesn’t allow the ALB’s security group on the health-check port. Health checks originate from the ALB ENIs; if the target SG’s ingress references a CIDR instead of the ALB SG id (or is missing), checks time out (Target.Timeout). Fix: add ingress from the ALB SG id. (SAA-C03, ANS-C01)

Q2. When does an ALB return 503 vs 502? 503 Service Unavailable means no healthy targets or no capacity — the ALB has nowhere to send the request. 502 Bad Gateway means it reached a target but got a malformed/closed response (often keep-alive shorter than the ALB idle timeout, or an app crash). 503 is a health/capacity problem; 502 is a target-response problem. (SAA-C03, SOA-C02)

Q3. How do you route /api/* to one fleet and everything else to another? One HTTPS listener, a rule at a low priority with condition path-pattern=/api/* forwarding to the API target group, and the listener’s default action forwarding to the web target group. Priority ordering matters — the specific rule must have a lower number than any catch-all. (SAA-C03)

Q4. Difference between duration-based and application-based stickiness? Duration-based uses ALB-generated cookies (AWSALB) with a fixed lifetime; application-based (AWSALBAPP) keys stickiness on your application cookie so affinity follows the app session. Both pin a client to a target; app-based aligns affinity with your session lifecycle. (SAA-C03, DVA-C02)

Q5. Why might one AZ carry far more load than another? Long stickiness durations pin established cohorts to their original targets, so scale-out adds cold targets that get little traffic; or cross-zone load balancing is disabled with uneven target counts per AZ. Fix: shorten stickiness / move state out; confirm cross-zone is on (it’s on and free by default for ALB). (ANS-C01)

Q6. What is deregistration delay and how do you size it? The window (default 300s, 0–3600) during which a deregistering target keeps draining in-flight requests before the ALB closes them. Size it to at least your longest legitimate request so deploys/scale-in don’t cut long operations; pair with graceful in-app shutdown. (SOA-C02)

Q7. How does an ALB serve multiple TLS certificates on one listener? Via SNI: one default certificate plus up to 25 total, and the ALB selects the cert matching the client’s SNI hostname. Clients that send no SNI receive the default cert. Certs come from ACM (free, auto-renewing) or IAM. (SAA-C03, SCS-C02)

Q8. Which target types can an ALB use, and when would you pick IP over instance? Instance (EC2 IDs), IP (any routable VPC/peered/on-prem IP), and Lambda. Pick IP for containers with awsvpc networking, for on-prem targets over Direct Connect/VPN, or when you need multiple ports per host — instance type registers the instance’s primary IP only. (SAA-C03, DVA-C02)

Q9. What causes a redirect loop behind an ALB and how do you fix it? TLS terminates at the ALB, which forwards plain HTTP; the app then redirects HTTP→HTTPS, which loops. Fix: the app must trust X-Forwarded-Proto and only redirect when it isn’t https. (SOA-C02)

Q10. How is ALB priced, and what is an LCU? A flat per-ALB-hour charge plus LCU-hours. An LCU is the maximum of four dimensions — new connections (25/s), active connections (3,000/min), processed bytes (1 GB/hr), rule evaluations (1,000/s, first 10 rules free). You pay for whichever dimension you hit hardest. (SAA-C03)

Q11. What’s the health-check “matcher” and when do you change it? The set of HTTP codes considered healthy (default 200; can be a list 200,301 or range 200-299; gRPC uses 0-99). Change it when your health endpoint legitimately returns something other than 200 (e.g., 301 for a redirecting root), or widen it to reduce false unhealthies. (SOA-C02)

Q12. How do you do a canary/blue-green deploy at the ALB with no DNS change? Use a weighted forward action listing both target groups with integer weights; shift weight from old to new (95/5 → 50/50 → 0/100) and optionally enable target-group stickiness so a session stays on one version. Instant and reversible. (DVA-C02, SAA-C03)

Quick check

1. Your ALB returns 503 on /api/* but the web store works. Which single command tells you why fastest?
2. Targets are unhealthy with reason Target.Timeout though the app runs. What’s the first thing to check?
3. You attach a second hostname’s certificate to the HTTPS listener but old browsers get the wrong cert. Why?
4. A deploy cuts users’ long file downloads mid-transfer. Which target-group attribute do you change?
5. Which is on-by-default and free on an ALB but off-by-default and billable on an NLB?

Answers

1. aws elbv2 describe-target-health --target-group-arn <api-tg> — a per-path 503 means that target group is empty; the reason code names the cause.
2. The target security group’s ingress — it must allow the ALB SG id on the traffic/health port. Target.Timeout on a healthy app is almost always the SG chain.
3. Those clients send no SNI, so they receive the listener’s default certificate, not the SNI cert you added. SNI selection needs the client to send the hostname in the handshake.
4. deregistration_delay.timeout_seconds — raise it to at least the longest legitimate request so draining doesn’t force-close in-flight transfers.
5. Cross-zone load balancing — on and free on ALB, off and inter-AZ-billable on NLB.

Glossary

Next steps

• Choose the right front door for each workload with AWS Load Balancers and API Gateway: ALB, NLB and API Gateway Compared — the layer-first decision guide upstream of this deep dive.
• See the ALB in a full stack in The Classic AWS Three-Tier Web Application Architecture, where it sits between the VPC edge and the Auto Scaling tier feeding RDS.
• Put resilient DNS in front of the ALB with Amazon Route 53 in Practice: Records, Alias & Routing Policies — alias records, health checks and failover between ALBs.
• When the ALB is throwing 5xx, work the dedicated incident guide, the ALB 502/503/504 troubleshooting playbook.
• Practice by adding a WAF Web ACL and an authenticate-oidc action to the lab’s HTTPS listener, then re-run the routing and health checks to confirm auth and filtering work end to end.