EC2 Auto Scaling Isn't Scaling: A Complete Troubleshooting Playbook

Quick take: an Auto Scaling group (ASG) does not “scale” — a pipeline scales, and every link can break with the identical symptom: nothing happens. A metric feeds a CloudWatch alarm, the alarm drives a scaling policy, the policy adjusts DesiredCapacity, the group asks EC2 to launch, and the new instance must survive a health check inside the grace period. When someone says “Auto Scaling is broken,” they have observed the output of a six-stage control loop and drawn a one-word conclusion. This playbook takes you stage by stage — with the exact describe-scaling-activities and describe-auto-scaling-groups output that tells you which link snapped — and fixes the number-one real cause, which throws no error at all.

You watch CPUUtilization sit at 94% for fifteen minutes. The dashboard is red. The ASG’s desired count does not move. No new instances. No alarm email. No entry anywhere that says why. To the on-call engineer this reads as a single failure — “the Auto Scaling group isn’t scaling” — but “not scaling” is not a root cause; it is a symptom shared by at least a dozen distinct faults, most of which are silent. The alarm might be in INSUFFICIENT_DATA. The group might already be at MaxSize. A cooldown from the last action might still be counting down. The launch might be failing on Insufficient Capacity in that Availability Zone. A process might be suspended from a half-finished instance refresh three days ago. Each of these produces the same view — a flat desired count — and each has a completely different fix.

This article is a diagnostic tree, not a feature tour. You will learn to treat the scaling control loop as a series of hops and to ask, at each one, “did the signal make it through?” — because the single most valuable skill here is knowing which of the six links broke before you touch a single setting. We cover every failure class end to end: the dead or mis-aggregated alarm, the silent already-at-max cap (the #1 cause), cooldown and warmup blocking the next action, target-tracking’s hidden internals, the whole family of launch failures (ICE, vCPU quota, full subnet, invalid AMI/key/instance-profile, Spot), the health-check kill loop that masquerades as “won’t scale,” and the process-level traps (suspended processes, scheduled actions in the wrong timezone, scale-in protection). The heart of the article is a symptom → root cause → confirm → fix playbook, an Activity-History status-code reference, and a decision table — plus a hands-on lab where you reproduce two of these failures on purpose and read the evidence AWS leaves behind.

What problem this solves

In production, “the ASG won’t scale” is one of the most expensive incidents to misdiagnose, because the symptom is so generic that engineers reach for the wrong fix by reflex. The classic waste pattern: CPU is pegged, so someone assumes the alarm is broken, rewrites the scaling policy, redeploys, and nothing changes — because the group was at MaxSize the whole time and no alarm could ever have helped. Hours evaporate on the wrong link in the chain.

The pain is worse than a slow dashboard. A group that won’t scale out during a traffic spike drops requests, blows the SLA, and can cascade into a full outage when the surviving instances tip over. A group that won’t scale in after the spike burns money on idle capacity — a stuck scale-in is a silent budget leak nobody notices until the bill. A group stuck in a launch-and-kill loop (health-check flapping) looks like “not scaling” but is actually thrashing EC2 launches every few minutes, generating cost, log noise, and a cycling fleet that never stabilises. And the nastiest, Insufficient Capacity, means AWS itself cannot give you the instance type you asked for in that AZ — no config change on your side fixes it; you need instance-type and AZ diversification, which is an architecture decision you’d rather have made before the incident.

Everyone who runs stateless compute on EC2 hits this: web tiers behind an ALB, batch workers pulling from SQS, ECS-on-EC2 capacity providers (which are ASGs underneath), game servers, CI runner fleets. The people who resolve it in five minutes instead of five hours are the ones who know the control loop has six links and know the two describe-* commands that show which one broke.

Learning objectives

By the end of this article you can:

• Model the scaling control loop as six hops (metric → alarm → policy → group → launch → health) and localise a “won’t scale” symptom to exactly one of them.
• Read describe-scaling-activities and describe-auto-scaling-groups as your primary evidence — an empty activity history, a Failed StatusCode, a SuspendedProcesses list, Desired == Max — and know what each proves.
• Diagnose a dead alarm: wrong metric/dimensions, threshold never crossed, period × evaluation-periods too slow, ALARM vs INSUFFICIENT_DATA, and a mis-chosen statistic that averages away the hot instance.
• Recognise the silent already-at-max cap — the #1 cause — and alarm on it so it is never silent again.
• Untangle target-tracking internals: the hidden AlarmHigh/AlarmLow it creates, why a too-high target never triggers, and how two policies fight.
• Classify every launch failure — ICE, VcpuLimitExceeded, InsufficientFreeAddressesInSubnet, invalid AMI/key/instance-profile, unavailable Spot — from the Activity-History status message, and apply the right fix (diversify, raise a quota, free IPs, fix the template).
• Break and repair the health-check kill loop by getting HealthCheckGracePeriod, the health-check type, and the health path right.
• Reproduce two failures in a lab (max=desired, and a bad instance profile), read the evidence, fix them, and tear it all down.

Prerequisites & where this fits

You should be comfortable launching an EC2 instance and reading its console, know what a launch template, a security group, a subnet, and an Availability Zone are, and have the AWS CLI v2 configured with permission to run autoscaling:Describe*, cloudwatch:DescribeAlarms, and ec2:Describe*. Familiarity with CloudWatch metrics and alarms and with an Application Load Balancer (target groups + health checks) will make the ELB-health sections click faster.

Where this sits: Auto Scaling is the elasticity layer of an EC2 architecture. Below it is the launch template (which instance, which AMI, which IAM role) — if the template is wrong, scaling can’t launch, so this article leans on EC2 Auto Scaling & Launch Templates Hands-On. Beside it are the service quotas that cap how many vCPUs you may launch — covered in AWS Service Quotas & Limits. In front of it is the load balancer whose health checks the ASG can honour — see ALB & Target Groups Hands-On. And when capacity is the constraint, the answer is often Spot + instance diversification — the wave sibling EC2 Spot Instances: Savings & Interruptions. This article is the troubleshooting capstone that ties them together: when scaling misbehaves, the fault is almost always in one of those adjacent layers, surfacing as a flat desired count.

Core concepts

Before you can debug the loop you have to see it as a loop. An ASG is a controller with three numbers, a set of policies that move the middle number, and a health system that replaces instances that die. Get these definitions crisp and every symptom maps to a moving part.

The three sizes. An ASG is bounded by MinSize (never fewer), MaxSize (never more), and steered to DesiredCapacity (the target it drives toward). Scaling is the act of changing DesiredCapacity — everything else is machinery to decide the new value and to make reality match it. The single most important invariant: DesiredCapacity is always clamped to [MinSize, MaxSize]. A policy can compute “go to 8,” but if MaxSize is 4, desired becomes 4 — silently.

The policy types. A scaling policy decides the new DesiredCapacity. There are five ways to drive it, and they behave very differently when something goes wrong.

The alarm. For step and simple scaling you create a CloudWatch alarm and attach the policy to it. For target tracking, AWS creates and manages the alarms for you (you never see them in your IaC, but they exist in the console). An alarm has three states — OK, ALARM, INSUFFICIENT_DATA — and only a transition into ALARM fires a scale-out policy. This is why a dead metric (which parks the alarm in INSUFFICIENT_DATA) means nothing scales, even though nothing is technically “broken.”

Health checks and the grace period. An ASG continuously checks each instance’s health and replaces any it deems unhealthy. The health-check type decides what “healthy” means: EC2 (the instance status checks) or ELB (the load balancer’s health check) or a custom signal. The HealthCheckGracePeriod is how long after launch the ASG ignores health results, giving the app time to boot. Get this too short with ELB health checks and you get the kill loop: launch → not-yet-ready → “unhealthy” → terminate → relaunch, forever.

Cooldown vs warmup. A cooldown (simple scaling) is a group-wide pause after any action during which no further action fires. An instance warmup (step + target tracking) is per-instance: a newly launched instance’s metrics are excluded from the aggregate until it has warmed up, so a cold instance’s 100% CPU doesn’t trigger a runaway scale-out. Confusing the two is a common source of “why did it only scale once?”

Suspended processes. An ASG runs nine internal processes (Launch, Terminate, HealthCheck, ReplaceUnhealthy, AZRebalance, AlarmNotification, ScheduledActions, AddToLoadBalancer, InstanceRefresh). Any can be suspended — by you, or automatically during an instance refresh. A suspended Launch means the group cannot add instances; a suspended AlarmNotification means alarms are ignored entirely. This is the trap that survives across incidents.

Activity history. Every attempt the ASG makes — successful or failed — is recorded as a scaling activity with a StatusCode, a StatusMessage, a Cause, and a Description. This is your black box. aws autoscaling describe-scaling-activities is the single most important command in this article: it tells you whether the group tried and failed (a Failed activity with the reason) or never tried at all (an empty history — look upstream at the alarm or the cap).

The commands you will live in during any scaling incident:

Instance lifecycle states. An instance inside an ASG moves through a lifecycle, and two of these states are quiet “won’t scale/serve” causes — an instance in Standby is still counted toward DesiredCapacity but receives no traffic and isn’t health-managed, and one stuck in Pending:Wait is being held by a lifecycle hook that never completed.

The scaling control loop, end to end

Here is the whole loop and, at each hop, the thing that can make the signal die. Debugging is nothing more than walking this table top to bottom and finding the first hop where the answer is “no.”

The very first fork — before you read any config — is this one question, answered by describe-scaling-activities:

Internalise that fork and you have already halved every future incident: “did it try?” An empty history sends you upstream to the trigger and the cap; a Failed history sends you downstream to the launch. People waste hours because they never ask it.

Failure class 1 — the CloudWatch alarm never fires

For step and simple scaling, a scale-out that never happens usually means the alarm never entered ALARM. There are more ways to misconfigure an alarm than any other single object in this loop, so enumerate them.

The two that fool experienced engineers are INSUFFICIENT_DATA and aggregation. INSUFFICIENT_DATA is not an error state — it means the alarm has no datapoints to evaluate, most often because the metric source disappeared (an instance terminated, the CloudWatch agent died, or the metric is only emitted under load and there’s currently none). A scale-out alarm parked in INSUFFICIENT_DATA will not fire, and the default TreatMissingData behaviour (missing) keeps it there. The aggregation trap is subtler: if you alarm on the Average CPUUtilization across an ASG dimension, one instance pinned at 100% while nine idle at 10% yields an average of 19% — far below any sane threshold — so the fleet “isn’t scaling” while one box melts. For uneven load, alarm on the Maximum, or on a per-request metric like ALB RequestCountPerTarget, not the fleet average.

The alarm’s TreatMissingData setting decides what a gap in datapoints does — and the default (missing) is exactly what parks a scale-out alarm in INSUFFICIENT_DATA when the metric goes quiet:

Reaction time is a function you can compute. Period × EvaluationPeriods is the minimum time an alarm needs breaching datapoints before it fires:

Create a proper scale-out alarm and policy with both CLI and Terraform. In CLI (step scaling):

# Step-scaling policy: +2 instances when it fires
POLICY_ARN=$(aws autoscaling put-scaling-policy \
  --auto-scaling-group-name web-asg \
  --policy-name scale-out-cpu \
  --policy-type StepScaling \
  --adjustment-type ChangeInCapacity \
  --step-adjustments MetricIntervalLowerBound=0,ScalingAdjustment=2 \
  --estimated-instance-warmup 120 \
  --query PolicyARN --output text)

# Alarm on the ASG-average CPU, 60s period, 2-of-3 → fires the policy
aws cloudwatch put-metric-alarm \
  --alarm-name web-asg-cpu-high \
  --namespace AWS/EC2 --metric-name CPUUtilization \
  --dimensions Name=AutoScalingGroupName,Value=web-asg \
  --statistic Average --period 60 \
  --evaluation-periods 3 --datapoints-to-alarm 2 \
  --threshold 70 --comparison-operator GreaterThanThreshold \
  --treat-missing-data notBreaching \
  --alarm-actions "$POLICY_ARN"

resource "aws_autoscaling_policy" "scale_out" {
  name                   = "scale-out-cpu"
  autoscaling_group_name = aws_autoscaling_group.web.name
  policy_type            = "StepScaling"
  adjustment_type        = "ChangeInCapacity"
  estimated_instance_warmup = 120
  step_adjustment {
    scaling_adjustment          = 2
    metric_interval_lower_bound = 0
  }
}

resource "aws_cloudwatch_metric_alarm" "cpu_high" {
  alarm_name          = "web-asg-cpu-high"
  namespace           = "AWS/EC2"
  metric_name         = "CPUUtilization"
  dimensions          = { AutoScalingGroupName = aws_autoscaling_group.web.name }
  statistic           = "Average"
  period              = 60
  evaluation_periods  = 3
  datapoints_to_alarm = 2
  threshold           = 70
  comparison_operator = "GreaterThanThreshold"
  treat_missing_data  = "notBreaching"
  alarm_actions       = [aws_autoscaling_policy.scale_out.arn]
}

Failure class 2 — already at max capacity (the silent #1)

This is the number-one cause and it is invisible. The policy fires correctly, computes a new DesiredCapacity of, say, 8, the ASG clamps it to MaxSize (4), and — critically — records nothing. No failed activity. No alarm. No log line saying “I wanted more but I’m capped.” The activity history for the spike window is simply empty, and the desired count sits pinned at max. Engineers stare at a working alarm and a working policy and conclude the machinery is broken, when the machinery did exactly what it was told: it cannot exceed MaxSize.

The clamp is pure arithmetic. Walk these examples and the silence makes sense:

Confirming it takes ten seconds — and it should be your second command after describe-scaling-activities comes back empty:

aws autoscaling describe-auto-scaling-groups \
  --auto-scaling-group-names web-asg \
  --query 'AutoScalingGroups[0].{Min:MinSize,Desired:DesiredCapacity,Max:MaxSize,InService:length(Instances)}'
# { "Min": 2, "Desired": 4, "Max": 4, "InService": 4 }   ← Desired == Max: this is the whole bug

The fix is to raise MaxSize — but the real fix is to make the cap never silent again by alarming on it. CloudWatch publishes GroupDesiredCapacity and GroupMaxSize in AWS/AutoScaling (you must enable group metrics collection); an alarm on GroupDesiredCapacity >= GroupMaxSize for a few minutes turns the #1 silent failure into a page.

# Raise the ceiling
aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name web-asg --max-size 10

# Enable group metrics so you can even see the cap
aws autoscaling enable-metrics-collection \
  --auto-scaling-group-name web-asg --granularity "1Minute"

resource "aws_autoscaling_group" "web" {
  name             = "web-asg"
  min_size         = 2
  desired_capacity = 4
  max_size         = 10            # headroom — not equal to desired
  enabled_metrics  = ["GroupDesiredCapacity", "GroupMaxSize", "GroupInServiceInstances"]
  # ...
}

A related, equally silent variant: MinSize blocking scale-in. If the group won’t shrink after the spike, check Desired > Min. When Desired == Min, scale-in is a no-op by design, and — like the max case — it records nothing. Half of “why won’t it scale in” tickets are simply Min set higher than anyone remembers.

Failure class 3 — cooldown and warmup block the next action

You watched it scale out once, then nothing, even though load kept climbing. That is usually cooldown (simple scaling) or instance warmup (step/target tracking) doing its job — suppressing the next action so a single spike doesn’t stampede the fleet. The failure is not that they exist; it’s that they’re set too long, or that you expected the wrong one.

The distinction that trips people: cooldown pauses the whole group (so simple scaling truly does nothing for 300s after any action), while warmup does not pause scaling — it just stops a cold, briefly-100%-CPU instance from being counted, so the aggregate metric settles correctly. If you’re on simple scaling and see exactly one action per five minutes, the cooldown is the cause; switch to target tracking or step scaling, which react continuously and use warmup instead of a blunt global pause.

# Simple-scaling policy carrying a 300s cooldown (the blocker)
aws autoscaling put-scaling-policy \
  --auto-scaling-group-name web-asg --policy-name simple-out \
  --policy-type SimpleScaling --adjustment-type ChangeInCapacity \
  --scaling-adjustment 2 --cooldown 60      # shorten to 60s, or move off simple scaling

# See "in cooldown" notes in the activity history
aws autoscaling describe-scaling-activities --auto-scaling-group-name web-asg \
  --query 'Activities[].{T:StartTime,S:StatusCode,C:Cause}' --output table

The default instance warmup is now taken from the group-level DefaultInstanceWarmup if set (recommended), overriding per-policy EstimatedInstanceWarmup. Set it to your app’s real boot-to-ready time. Too short and cold instances skew the metric (causing over-scaling); too long and the group is sluggish to react to genuine, sustained load.

The default timings and where each is configured, so you know which knob you’re actually turning:

Failure class 4 — target-tracking internals

Target tracking feels like magic — “keep CPU at 50%” — precisely because it hides its machinery, and hidden machinery is hard to debug. Three things you cannot see are the usual culprits.

It creates hidden alarms. When you attach a target-tracking policy, AWS silently creates two CloudWatch alarms in your account — a high alarm to scale out and a low alarm to scale in — named like TargetTracking-web-asg-AlarmHigh-<uuid>. You did not write them, they aren’t in your Terraform, but they run your scaling. If someone deletes them by hand, or a cleanup script prunes “orphan” alarms, scaling quietly dies.

A too-high target never triggers. Target tracking scales out to pull the metric down to the target. If you set the target to CPUUtilization = 90%, the group only adds capacity when CPU exceeds 90% — so a fleet cruising at 85% is, by design, “at target” and will never scale out, even though it’s clearly hot. Target tracking also scales out faster than it scales in (it’s cautious about removing capacity), so a too-high target combined with steady load looks exactly like “won’t scale.”

Two policies fighting. If you attach two target-tracking policies on correlated metrics — say CPU and network — they can issue contradictory desired counts and the group oscillates or stalls. AWS’s rule is that when multiple policies apply, the ASG picks the one that yields the largest capacity for scale-out (safe) but this still produces confusing behaviour when policies disagree on scale-in. Prefer one dominant metric per group.

The predefined metrics you can target-track on — pick the one that actually reflects your bottleneck:

resource "aws_autoscaling_policy" "cpu_tt" {
  name                   = "cpu-target-50"
  autoscaling_group_name = aws_autoscaling_group.web.name
  policy_type            = "TargetTrackingScaling"
  target_tracking_configuration {
    predefined_metric_specification {
      predefined_metric_type = "ASGAverageCPUUtilization"
    }
    target_value     = 50      # NOT 90 — too high never triggers
    disable_scale_in = false   # true here is a frequent "won't scale in" cause
  }
}

For an SQS-worker fleet, target-track on a custom metric like ApproximateNumberOfMessagesVisible / instance (backlog per worker), because CPU may stay low while the queue explodes — a textbook “CPU-based scaling won’t scale” bug.

Failure class 5 — the launch fails

Now the downstream half. DesiredCapacity went up — you can see it rise — but the InService count doesn’t follow, and describe-scaling-activities shows StatusCode = Failed. EC2 refused to launch. The StatusMessage/Cause names the reason exactly; this is the reference table to match it against.

Three of these deserve emphasis. ICE (Insufficient Capacity) is the one you cannot fix with configuration on your side — AWS genuinely has no spare m5.24xlarge in ap-south-1a at that moment. The engineering answer is diversification: an ASG with a mixed instances policy across several instance types and all AZs, using a capacity-optimized allocation strategy, so the launch succeeds on whatever is available instead of insisting on one type in one AZ. vCPU quota looks identical from the app’s view (instances just don’t appear) but the StatusMessage says “vCPU limit”; the fix is a quota-increase request, and the prevention is a CloudWatch alarm on the AWS/Usage vCPU metric at ~80% of the limit. Invalid instance profile is the launch failure engineers cause most often themselves: they delete or rename an IAM role a launch template still references, or forget iam:PassRole, and every launch fails instantly — you’ll reproduce exactly this in the lab.

The ICE fix as a mixed-instances ASG:

resource "aws_autoscaling_group" "web" {
  name                = "web-asg"
  min_size            = 2
  desired_capacity    = 4
  max_size            = 12
  vpc_zone_identifier = [aws_subnet.a.id, aws_subnet.b.id, aws_subnet.c.id]  # all AZs

  mixed_instances_policy {
    instances_distribution {
      on_demand_base_capacity                  = 2
      on_demand_percentage_above_base_capacity = 50
      spot_allocation_strategy                 = "capacity-optimized"   # dodges ICE
    }
    launch_template {
      launch_template_specification {
        launch_template_id = aws_launch_template.web.id
        version            = "$Latest"
      }
      override { instance_type = "m5.large" }
      override { instance_type = "m5a.large" }   # diversify family
      override { instance_type = "m6i.large" }
      override { instance_type = "m5n.large" }
    }
  }
}

When you diversify to dodge ICE, the allocation strategy decides how the ASG picks from your instance-type overrides. Capacity-optimized (or price-capacity-optimized) is the ICE-avoidance default because it launches where capacity actually exists:

Failure class 6 — the health-check kill loop

This one masquerades. The group is launching — repeatedly — but every new instance is terminated within a minute or two and relaunched, so the InService count never grows and it presents as “won’t scale.” The activity history is the giveaway: alternating Launching and Terminating entries, the terminations reading “instance failed ELB health checks.”

The mechanism: you set HealthCheckType = ELB (so the ASG honours the load balancer’s opinion of health), but the HealthCheckGracePeriod is shorter than the app’s real boot time. The instance launches, the ALB health check hits /healthz before the app is listening, marks it unhealthy, and the ASG — no longer in the grace window — dutifully replaces the “unhealthy” instance. Boot, fail, kill, repeat.

The two health-check types behave very differently, and choosing ELB without a matching grace period is the trap:

The kill loop is really a race between two clocks — the app’s boot-to-healthy time and the health check’s time-to-unhealthy (Interval × UnhealthyThreshold). These are the tunables on both sides of that race:

# The fix is almost always: raise the grace period above real boot time
aws autoscaling update-auto-scaling-group \
  --auto-scaling-group-name web-asg \
  --health-check-type ELB \
  --health-check-grace-period 300      # was 60s and shorter than a 150s boot

resource "aws_autoscaling_group" "web" {
  # ...
  health_check_type         = "ELB"
  health_check_grace_period = 300   # MUST exceed boot-to-first-healthy time
}

The subtle version: EC2 health checks are too lenient (a wedged app on a running box is “healthy,” so the ASG never replaces it and the ALB serves errors), while ELB health checks with a short grace are too strict (the kill loop). The correct posture is ELB health checks plus a grace period comfortably above your measured boot time, and a health endpoint that returns 200 only when the app is genuinely ready.

Failure class 7 — schedules, protection, termination and suspended processes

The last family isn’t about metrics at all — it’s about the group’s governance settings quietly overriding or disabling scaling.

Suspended processes. Any of nine processes can be suspended, and a suspended process is a silent kill switch. The most damaging are Launch, Terminate, and AlarmNotification.

The trap is that an instance refresh automatically suspends certain processes while it runs, and a refresh that fails or is cancelled mid-flight can leave them suspended. Days later, scaling is dead and nobody remembers the refresh. Always check:

aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names web-asg \
  --query 'AutoScalingGroups[0].SuspendedProcesses'
# [] means none suspended. Anything here is a prime suspect.

# Resume the ones you didn't mean to suspend
aws autoscaling resume-processes --auto-scaling-group-name web-asg \
  --scaling-processes Launch Terminate AlarmNotification

Scheduled actions override Min/Max/Desired at a point in time, and their two classic bugs are timezone and ordering/pinning. A scheduled action with no TimeZone runs in UTC — so an action meant for 8 a.m. IST fires at 1:30 p.m. IST. And a scheduled action that sets MinSize = 10 will pin desired at 10 and block any scale-in below it until the next action changes Min.

resource "aws_autoscaling_schedule" "morning_bump" {
  scheduled_action_name  = "morning-bump"
  autoscaling_group_name = aws_autoscaling_group.web.name
  recurrence             = "0 8 * * MON-FRI"
  time_zone              = "Asia/Kolkata"   # without this it's UTC
  min_size               = 2                 # keep Min low
  desired_capacity       = 8                 # bump Desired, not Min
  max_size               = 12
}

Scale-in protection and termination policy. Instance scale-in protection stops the ASG from terminating a protected instance during scale-in; if every instance is protected (e.g. NewInstancesProtectedFromScaleIn = true set group-wide), scale-in can do nothing and desired stays high. The termination policy decides which instance to remove on scale-in; a misordered policy won’t block scale-in outright but can produce surprising choices (killing your newest instance first).

Lifecycle hooks. One more governance trap sits between launch and InService: a lifecycle hook pauses an instance in Pending:Wait (or Terminating:Wait) until your automation signals CONTINUE. A hook whose automation never answers stalls scaling until the heartbeat times out, then abandons or continues per its default result — a launch that looks frozen for up to an hour.

Reading the evidence: Activity History status codes

Everything above converges on one artefact: the scaling activity. Learn to read its StatusCode and you can classify any failure without guessing. These are the values you’ll see:

A Failed activity carries the exact reason. Match its StatusMessage/Cause to this failure-code reference (the same causes as class 5, indexed for fast lookup during an incident):

Pull the last failures in one command and read them like a log:

aws autoscaling describe-scaling-activities \
  --auto-scaling-group-name web-asg --max-items 10 \
  --query 'Activities[].{Time:StartTime,Status:StatusCode,Msg:StatusMessage,Cause:Cause}' \
  --output table

Architecture at a glance

The diagram traces the real control loop left to right — metric → CloudWatch alarm → scaling policy → Auto Scaling group → launch attempt → instance — and drops a numbered badge at the exact hop each failure class bites. Follow the flow: a metric (CloudWatch, aggregated per-ASG) feeds an alarm whose failure to enter ALARM is badge 1; the alarm drives a scaling policy whose cooldown/warmup is badge 3; the policy raises the group’s DesiredCapacity, where the silent already-at-max cap is badge 2 and a suspended process is badge 6; the group asks EC2 to launch, where ICE, vCPU quota, a full subnet and a bad template are badge 4; and the new instance must pass a health check inside the grace period — the kill loop is badge 5. The legend narrates each number as symptom · confirm · fix, so the picture is both the architecture and the diagnostic map. The single most important reading habit the diagram encodes: run describe-scaling-activities first to learn whether the group tried and failed (badges 4–5, downstream) or never tried (badges 1–3, 6, upstream).

Real-world scenario

Trailhead Logistics runs a fleet of route-optimisation workers on an ASG behind an internal ALB in ap-south-1, normally 6 c5.xlarge instances, MaxSize 20. On the morning of a national sale, order volume tripled in twenty minutes. The dashboards lit up: queue depth climbing, CPU on every worker pinned near 100%. The ASG’s desired count moved from 6 to 9 — and then stopped. Requests started timing out. The on-call declared a Sev-2 and, reasonably, assumed the scaling policy was too timid, so they lowered the target-tracking target from 60% to 40% and redeployed. Desired didn’t budge. Twenty more minutes gone.

The engineer who fixed it ran the two commands in order. First, describe-scaling-activities — and it was not empty: it showed three Failed activities, each stamped We currently do not have sufficient c5.xlarge capacity in the Availability Zone you requested (ap-south-1a). This was ICE, not a policy problem. The group was pinned in a single AZ (a legacy vpc_zone_identifier with only one subnet) and hard-wired to one instance type. AWS simply had no more c5.xlarge in ap-south-1a during the region-wide sale spike. No target change on Earth would have helped — the launches were failing at the EC2 layer.

The immediate mitigation was surgical: they added the two other AZ subnets to the group (update-auto-scaling-group --vpc-zone-identifier with all three) and switched to a mixed instances policy overriding c5.xlarge, c5a.xlarge, c6i.xlarge, and m5.xlarge with a capacity-optimized allocation strategy. Within four minutes the next scaling activity succeeded — on c5a.xlarge in ap-south-1b, whatever had capacity — and desired climbed to 17 as the backlog drained.

The post-mortem changed three things. They made multi-AZ + multi-type the default for every worker ASG, because single-AZ single-type is an ICE incident waiting for a busy day. They added a CloudWatch alarm on failed scaling activities (via an EventBridge rule on the EC2 Instance Launch Unsuccessful event) so the next ICE pages with the cause attached instead of presenting as “won’t scale.” And they wrote the fork into the runbook as step one: describe-scaling-activities first — empty means look up at the alarm and the cap; Failed means read the EC2 reason. The three-hour outage would have been a five-minute diversification the moment someone read the activity history.

Advantages and disadvantages

Auto Scaling’s design — a decoupled control loop with rich activity history — is exactly what makes it both powerful and confusing to debug. The signals are all there; they’re just spread across six objects.

The through-line: Auto Scaling is observable to the point of being self-diagnosing — but only if you read the activity history and the group’s own numbers first, instead of assuming “not scaling” means “the policy is wrong.”

Hands-on lab

You will reproduce two real “won’t scale” failures on purpose, read the exact evidence AWS records, and fix each — then tear it all down. First the silent already-at-max cap (the #1 cause), then a launch failure via a deliberately broken IAM instance profile, reading the Failed activity. This is free-tier-friendly (one t3.micro at a time), but ⚠️ running instances cost money — do the teardown at the end.

Step 0 — Prerequisites

aws sts get-caller-identity
export AWS_REGION=ap-south-1
export VPC=vpc-0abc...                          # a VPC
export SUBNETS="subnet-0aaa,subnet-0bbb"        # two subnets in two AZs
export AMI=$(aws ssm get-parameters --names \
  /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
  --query 'Parameters[0].Value' --output text)
echo "AMI=$AMI"

Step 1 — A launch template (valid, to start)

LT_ID=$(aws ec2 create-launch-template --launch-template-name asg-lab-lt \
  --launch-template-data "{\"ImageId\":\"$AMI\",\"InstanceType\":\"t3.micro\"}" \
  --query 'LaunchTemplate.LaunchTemplateId' --output text)
echo "LT_ID=$LT_ID"

Step 2 — An ASG with the trap baked in: max == desired

aws autoscaling create-auto-scaling-group \
  --auto-scaling-group-name asg-lab \
  --launch-template "LaunchTemplateId=$LT_ID,Version=\$Latest" \
  --min-size 1 --desired-capacity 2 --max-size 2 \
  --vpc-zone-identifier "$SUBNETS"
aws autoscaling enable-metrics-collection \
  --auto-scaling-group-name asg-lab --granularity "1Minute"

# Wait for the two instances to come InService
aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names asg-lab \
  --query 'AutoScalingGroups[0].{Desired:DesiredCapacity,Max:MaxSize,InService:length(Instances)}'
# { "Desired": 2, "Max": 2, "InService": 2 }

Step 3 — Try to scale out, and watch the silence

# Ask for +2 via a simple set-desired above Max
aws autoscaling set-desired-capacity \
  --auto-scaling-group-name asg-lab --desired-capacity 4 || true

Expected: the command fails with New desired capacity 4 is above max capacity 2 if you set it directly. A policy wouldn’t error — it would clamp silently. Simulate the silent path by executing a scaling policy instead:

POLICY_ARN=$(aws autoscaling put-scaling-policy --auto-scaling-group-name asg-lab \
  --policy-name lab-out --policy-type SimpleScaling \
  --adjustment-type ChangeInCapacity --scaling-adjustment 2 \
  --query PolicyARN --output text)
aws autoscaling execute-policy --auto-scaling-group-name asg-lab --policy-name lab-out

# Now read the evidence: the activity history for the "scale-out"
aws autoscaling describe-scaling-activities --auto-scaling-group-name asg-lab \
  --max-items 5 --query 'Activities[].{S:StatusCode,C:Cause}' --output table

Expected: no new launch activity for the executed policy — the desired was already at Max (2), so it clamped to 2 and did nothing. describe-auto-scaling-groups still shows Desired: 2, Max: 2. This is the silent #1 cause reproduced. You have a working policy, a working execute call, and zero scaling — because of one number.

Step 4 — Fix it: raise MaxSize

aws autoscaling update-auto-scaling-group --auto-scaling-group-name asg-lab --max-size 4
aws autoscaling execute-policy --auto-scaling-group-name asg-lab --policy-name lab-out

aws autoscaling describe-scaling-activities --auto-scaling-group-name asg-lab \
  --max-items 3 --query 'Activities[].{S:StatusCode,D:Description}' --output table
# Now you see: Launching a new EC2 instance ...  (Desired went 2 -> 4)

Step 5 — Reproduce a launch failure: a bad instance profile

Create a new launch-template version that references an IAM instance profile that does not exist, then scale and read the Failed activity.

# A launch-template version pointing at a non-existent instance profile
aws ec2 create-launch-template-version --launch-template-id $LT_ID \
  --source-version 1 \
  --launch-template-data "{\"IamInstanceProfile\":{\"Name\":\"does-not-exist-profile\"}}"
aws ec2 modify-launch-template --launch-template-id $LT_ID --default-version 2

# Force a launch with the broken version: raise desired
aws autoscaling update-auto-scaling-group --auto-scaling-group-name asg-lab \
  --max-size 6 --desired-capacity 6

Wait ~60s, then read the Activity History:

aws autoscaling describe-scaling-activities --auto-scaling-group-name asg-lab \
  --max-items 5 \
  --query 'Activities[?StatusCode==`Failed`].{S:StatusCode,M:StatusMessage}' --output table

Expected: one or more Failed activities with a StatusMessage naming the bad profile — e.g. Invalid IAM Instance Profile name. DesiredCapacity shows 6, but the InService count is stuck at the instances launched on the good version. This is a launch failure reproduced, and you read the exact cause from the activity history — the same technique that solves an ICE or a quota incident.

Step 6 — Fix and confirm

# Point the default version back to the good one
aws ec2 modify-launch-template --launch-template-id $LT_ID --default-version 1
# New launches now use version 1 (no bad profile). Watch it recover:
aws autoscaling describe-scaling-activities --auto-scaling-group-name asg-lab \
  --max-items 5 --query 'Activities[].{S:StatusCode,D:Description}' --output table

Step 7 — Teardown ⚠️

aws autoscaling delete-auto-scaling-group --auto-scaling-group-name asg-lab \
  --force-delete                                   # terminates instances too
aws ec2 delete-launch-template --launch-template-id $LT_ID

Everything here is inside the free-tier compute envelope if you delete promptly, but instances billed per-second while running are the only cost — the --force-delete above terminates them.

The whole ASG + policy + launch template as Terraform, so you can reproduce the max-cap trap declaratively and flip it:

resource "aws_launch_template" "lab" {
  name_prefix   = "asg-lab-"
  image_id      = data.aws_ssm_parameter.al2023.value
  instance_type = "t3.micro"
}

resource "aws_autoscaling_group" "lab" {
  name                = "asg-lab"
  min_size            = 1
  desired_capacity    = 2
  max_size            = 2                # the trap: == desired. Change to 4 to fix.
  vpc_zone_identifier = var.subnet_ids
  launch_template {
    id      = aws_launch_template.lab.id
    version = "$Latest"
  }
  enabled_metrics = ["GroupDesiredCapacity", "GroupMaxSize", "GroupInServiceInstances"]
}

What each step of the lab proved, mapped back to the failure classes so the technique — not just the commands — transfers to a real incident:

Common mistakes & troubleshooting

This is the playbook — keep it open during an incident. Each row is symptom → root cause → confirm (the exact command/field) → fix. Work it top-down; the first rows are the most common and the cheapest to check.

Beyond the table, prose on the three that eat the most hours.

The silent max cap (badge 2). The most expensive minutes in any scaling incident are the ones spent debugging a policy that works perfectly. When DesiredCapacity == MaxSize, the policy computes a bigger number, the ASG clamps it, and nothing is recorded — no failed activity, no alarm, no log. Engineers see a healthy alarm firing and a healthy policy and conclude the whole system is broken, when it did exactly what it’s designed to do. The tell is always the same: describe-scaling-activities is empty for the spike window while desired sits pinned at max. Make it your second command, and permanently defeat it with an alarm on GroupDesiredCapacity >= GroupMaxSize.

Insufficient Capacity, ICE (badge 4). This is the failure you cannot fix by changing your configuration, because the constraint is on AWS’s side: there is genuinely no more of that instance type in that Availability Zone at that moment, usually during a regional demand spike. The activity history is explicit — “We currently do not have sufficient <type> capacity” — but only if you read it; from the app’s perspective it’s identical to every other “won’t scale.” Chasing it with policy changes is futile. The only durable answer is diversification: a mixed-instances policy spanning several instance types across every AZ with a capacity-optimized allocation strategy, so a launch succeeds on whatever is available. Single-type, single-AZ groups are ICE incidents waiting for a busy day.

The health-check kill loop (badge 5). This one hides as “won’t scale” while the group is frantically trying to scale — launching an instance every couple of minutes and terminating it just as fast. The cause is HealthCheckType = ELB with a HealthCheckGracePeriod shorter than the app’s real boot time: the instance launches, the ALB health check hits it before the app is listening, marks it unhealthy, and the ASG replaces the “failing” instance the moment the grace window closes. The activity history shows the tell-tale alternation of Launching and Terminating … failed ELB health checks a fixed number of seconds apart. The fix is almost never the scaling policy — it’s raising the grace period above your measured boot-to-healthy time and making sure the health endpoint returns 200 only when the app is truly ready.

The decision table for the first sixty seconds of an incident:

Best practices

• Run describe-scaling-activities first, always. It forks the entire investigation into “never tried” (upstream: alarm, cap, suspension) versus “tried and failed” (downstream: launch). Skipping this is why incidents run long.
• Alarm on the silent #1. Enable group metrics and alarm on GroupDesiredCapacity >= GroupMaxSize so an at-max cap pages you instead of hiding.
• Never set MaxSize == DesiredCapacity in steady state — always leave headroom, or scale-out is capped from the first spike.
• Prefer target tracking with a realistic target (40–60%) over simple scaling; it reacts continuously and uses warmup instead of a blunt cooldown.
• Diversify to survive ICE: mixed-instances policy across multiple instance types and every AZ, with a capacity-optimized allocation strategy. Single-type single-AZ is fragile.
• Set HealthCheckGracePeriod above measured boot time when using ELB health checks, and point the health check at a lightweight /healthz that only returns 200 when truly ready.
• Alarm on failed scaling activities via an EventBridge rule (EC2 Instance Launch Unsuccessful) so ICE/quota/template failures page with the cause attached.
• Alarm on vCPU usage at 80% of the quota (AWS/Usage) so a quota-driven launch failure is predicted, not discovered.
• Audit SuspendedProcesses after every instance refresh — a failed or cancelled refresh can leave Launch/Terminate suspended for days.
• Set TimeZone on every scheduled action and change DesiredCapacity, not MinSize, for temporary bumps so you don’t pin the floor.
• Use DefaultInstanceWarmup at the group level, set to real boot-to-ready time, so cold instances don’t skew the metric and cause over-scaling.
• Keep one dominant scaling metric per group — two fighting target-tracking policies oscillate.

The alarm/observability set that makes a “won’t scale” page arrive with the diagnosis attached:

Security notes

• Split diagnostic from remediation permissions. The on-call needs autoscaling:Describe*, cloudwatch:DescribeAlarms, and ec2:Describe* to read the loop — none of which can change capacity. Reserve autoscaling:UpdateAutoScalingGroup, SetDesiredCapacity, ResumeProcesses, and iam:PassRole for a separate remediation role. A read-only diagnostic role resolves most incidents without any write access.
• iam:PassRole is the sharp edge. For an ASG to launch instances with an instance profile, the principal creating/updating the group needs iam:PassRole for that role. Scope it to the specific role ARN with a Condition on iam:PassedToService = ec2.amazonaws.com — never Resource: "*", which lets a caller attach any role (including admin) to a new instance and pivot.
• The service-linked role is required. Auto Scaling uses the AWSServiceRoleForAutoScaling service-linked role to call EC2/ELB/CloudWatch on your behalf. If it’s missing or its permissions are tampered with (e.g. it can’t use a KMS key for an encrypted AMI), launches fail with opaque Client.InternalError — a security-adjacent failure that presents as “won’t scale.”
• Least-privilege launch templates. The instance profile a launch template attaches should carry only what the workload needs; a scaled fleet multiplies any over-permissioned role across every instance. Review the profile before it scales to dozens of boxes.
• Protect the scaling configuration from drift. Someone hand-editing MaxSize, suspending a process, or deleting a target-tracking alarm in the console is a common cause of silent scaling death; enforce the config via Terraform and detect drift, and log all autoscaling:* mutations in CloudTrail so “who set Max to Desired?” has an answer.
• Guard scheduled actions. A scheduled action that raises MinSize is effectively a standing capacity floor (and cost floor); restrict who can create them and review recurrences so an errant schedule can’t pin a large, expensive fleet.

Cost & sizing

Auto Scaling itself is free — you pay only for the EC2 instances, plus small charges for the CloudWatch metrics and alarms that keep it observable. But the failures in this article have real cost signatures worth knowing.

Sizing guidance to avoid capacity-driven failures:

The single biggest hidden cost here isn’t the ASG — it’s a group that won’t scale in because MinSize is too high, DisableScaleIn is set, or every instance is scale-in-protected. It runs a full fleet through every quiet night and weekend, and because “extra capacity” never pages anyone, it can leak for months. Alarm on GroupInServiceInstances staying above expected off-peak levels, not just on scale-out failures.

Interview & exam questions

1. An ASG won’t scale out during a spike; the alarm and policy look correct. What is the single most likely cause and how do you confirm it in ten seconds? The group is already at MaxSize — the #1 silent cause. The policy computes a higher desired, the ASG clamps it to Max, and records nothing. Confirm with describe-auto-scaling-groups and compare DesiredCapacity to MaxSize; if equal, raise MaxSize. (SAA-C03, SOA-C02)

2. What is the first command you run in any “won’t scale” incident and what does its output fork into? aws autoscaling describe-scaling-activities. An empty history means the group never tried (look upstream: dead alarm, at-max cap, or a suspended process); a Failed activity means it tried and EC2 refused (read the StatusMessage for ICE/quota/subnet/template). (SOA-C02)

3. Explain Insufficient Capacity (ICE) and why no configuration change on your side fixes it. ICE means AWS has no more of the requested instance type in the requested AZ at that moment. The launch fails with “We currently do not have sufficient <type> capacity.” Because the constraint is on AWS’s capacity, the durable fix is diversification — a mixed-instances policy across multiple types and all AZs with a capacity-optimized allocation strategy. (SAA-C03, ANS-C01)

4. A target-tracking policy set to keep CPU at 90% never scales out under obvious load. Why? Target tracking adds capacity only to pull the metric down to the target. At a 90% target, a fleet cruising at 85% is “at target,” so it never scales out even though it’s clearly hot. Lower the target to a realistic 40–60%. (SAA-C03)

5. Distinguish cooldown from instance warmup. Cooldown (simple scaling) is a group-wide pause after any action during which no further action fires (default 300s). Instance warmup (step/target tracking) is per-instance — it excludes a newly launched instance’s metrics from the aggregate while it boots, so a cold instance’s spike doesn’t cause runaway scaling. Cooldown blocks scaling; warmup doesn’t. (SOA-C02)

6. An instance launches and is terminated ~60 seconds later, repeatedly, and the group “won’t scale.” What’s happening and what’s the fix? The health-check kill loop: HealthCheckType = ELB with a HealthCheckGracePeriod shorter than the app’s boot time, so the ALB marks the not-yet-ready instance unhealthy and the ASG replaces it. Fix by raising the grace period above real boot time and correcting the health path/matcher. (SOA-C02, ANS-C01)

7. Where does target tracking store the logic that actually triggers scaling, and how can it silently break? In two CloudWatch alarms AWS creates and manages (TargetTracking-<asg>-AlarmHigh/AlarmLow-*), not in your IaC. If a cleanup script or a person deletes those alarms, scaling dies silently; re-putting the policy recreates them. (SAA-C03)

8. Name three ways a suspended process manifests as “won’t scale.” Suspended Launch → can’t add instances; suspended Terminate → can’t scale in or replace unhealthy; suspended AlarmNotification → alarms ignored so dynamic scaling is entirely dead. Confirm with describe-auto-scaling-groups → SuspendedProcesses; a failed instance refresh is a common cause. (SOA-C02)

9. A scaling activity shows Failed with “You have requested more vCPU capacity than your current vCPU limit.” What is this and how do you prevent recurrence? A service-quota (vCPU) limit for that instance family. Request a quota increase via Service Quotas, and prevent recurrence by alarming on the AWS/Usage vCPU metric at ~80% of the limit so you raise it before the peak. (SOA-C02, CLF-C02)

10. Why might an alarm on Average CPUUtilization across an ASG fail to scale a fleet where one instance is pegged at 100%? Averaging hides the hot instance — one node at 100% among nine at 10% averages 19%, below any sane threshold. For uneven load, alarm on Maximum or on a per-target metric like ALB RequestCountPerTarget. (SAA-C03)

11. A group scales out fine but never scales in. Give three distinct causes. DesiredCapacity == MinSize (Min too high); target-tracking DisableScaleIn = true; or scale-in protection (NewInstancesProtectedFromScaleIn) on all instances. Each blocks scale-in independently. (SOA-C02)

12. What does INSUFFICIENT_DATA on a scale-out alarm mean for scaling, and what causes it? The alarm has no datapoints to evaluate, so it can’t transition to ALARM and nothing scales. Common causes: the metric source disappeared (instance terminated, agent died) or the metric only publishes under load. Fix the source or set TreatMissingData appropriately. (SOA-C02)

Quick check

1. What is the first command you run in a “won’t scale” incident, and what do an empty history versus a Failed activity each tell you?
2. The #1 silent cause of no scale-out is what condition between two of the three ASG sizes, and how do you confirm it?
3. Why does a target-tracking target of 90% often “never scale out,” and what should it be?
4. An instance launches then terminates ~60s later on repeat — name the failure and its one-line fix.
5. Which describe-auto-scaling-groups field reveals a process-level kill switch, and name the three most damaging processes to have suspended?

Answers

1. aws autoscaling describe-scaling-activities. Empty = the group never tried to scale (look upstream: dead alarm, at-max cap, or a suspended process). Failed = it tried and EC2 refused (read the StatusMessage for ICE, vCPU quota, full subnet, or a bad template).
2. DesiredCapacity == MaxSize. The policy computes a higher desired, the ASG clamps it to Max, and records nothing. Confirm with describe-auto-scaling-groups and compare DesiredCapacity to MaxSize; the fix is to raise MaxSize.
3. Target tracking only adds capacity to pull the metric down to the target, so at 90% a fleet at 85% is “at target” and won’t scale out. Set a realistic target of 40–60%.
4. The health-check kill loop — HealthCheckType = ELB with a HealthCheckGracePeriod shorter than boot time. Fix: raise the grace period above real boot-to-healthy time (and fix the health path).
5. SuspendedProcesses. The three most damaging: Launch (can’t add), Terminate (can’t remove/replace), and AlarmNotification (alarms ignored — dynamic scaling entirely dead). Resume with resume-processes.

Glossary

• Auto Scaling group (ASG) — a managed collection of EC2 instances kept between MinSize and MaxSize and steered to DesiredCapacity by policies and health checks.
• DesiredCapacity — the target instance count the ASG drives toward; always clamped to [MinSize, MaxSize]. Scaling is changing this number.
• Scaling activity — a recorded attempt to change capacity, with a StatusCode (Successful/Failed/…), StatusMessage, and Cause; the primary evidence in any incident.
• Target tracking — a policy that keeps a metric at a target value using two CloudWatch alarms AWS creates and manages for you.
• Step scaling — a policy that maps metric bands to step adjustments; uses instance warmup rather than a global cooldown.
• Simple scaling — a legacy policy: one alarm, one adjustment, then a group-wide cooldown (default 300s) that blocks all actions.
• Cooldown — a post-action, group-wide pause (simple scaling) during which no further scaling happens.
• Instance warmup — a per-instance period during which a new instance’s metrics are excluded from the aggregate so a cold instance’s spike doesn’t cause over-scaling.
• Health-check grace period — seconds after launch during which the ASG ignores health results; too short with ELB health checks causes the kill loop.
• Insufficient Capacity / ICE — AWS has no more of the requested instance type in the requested AZ right now; fixed by diversifying types and AZs, not by config.
• Mixed instances policy — an ASG configuration spanning multiple instance types (and Spot/On-Demand) with an allocation strategy (e.g. capacity-optimized) to survive ICE.
• Suspended process — one of nine ASG processes (Launch, Terminate, AlarmNotification, …) turned off, silently disabling part or all of scaling.
• Scale-in protection — a flag that stops the ASG from terminating specific (or all new) instances during scale-in.
• INSUFFICIENT_DATA — a CloudWatch alarm state meaning no datapoints to evaluate; a scale-out alarm stuck here never fires.
• iam:PassRole — the permission a principal needs to let an ASG launch instances with an instance profile; a missing grant is a frequent launch-failure cause.

Next steps

• Build the fleet you just learned to debug, template-first: EC2 Auto Scaling & Launch Templates Hands-On.
• Fix the launch failures that come from limits: AWS Service Quotas & Limits: Finding Them, Requesting Increases & Avoiding Throttling.
• Get the health checks the ASG honours right at the load-balancer layer: Application Load Balancer & Target Groups Hands-On, and when it’s a 503 not a scaling issue, the ALB 5xx troubleshooting playbook.
• Beat Insufficient Capacity with diversification and cheaper compute: EC2 Spot Instances: Savings & Interruptions.
• When a launched instance is running but unreachable (not a scaling problem), work the EC2 SSH connect-timeout playbook and the VPC connectivity troubleshooting playbook.