Amazon SQS Hands-On: Standard vs FIFO Queues, Visibility Timeout & DLQs

Two services that must talk to each other should almost never talk directly. The moment your order API calls your fulfilment worker synchronously, you have welded them together: if the worker is slow, the API is slow; if the worker is down, the order is lost; if a sale sends ten thousand orders a second, the worker either falls over or you over-provision it for a peak that lasts four minutes a year. Amazon SQS (Simple Queue Service) is the shock absorber that removes that weld. The producer drops a message into a queue and moves on; the consumer pulls messages when it is ready, at its pace; and the queue holds the backlog — for up to 14 days — so a slow or crashed consumer costs you latency, not data.

SQS is a fully managed, effectively infinite message queue: no brokers to patch, no disks to size, no cluster to scale. You create a queue, you SendMessage, you ReceiveMessage, you DeleteMessage — that is nearly the whole API surface. But hiding behind that tiny surface are the four concepts that decide whether your system is correct or quietly broken, and this article installs all four by having you build the thing. Standard queues give you near-unlimited throughput but at-least-once delivery — meaning the same message can arrive twice, and messages can arrive out of order. FIFO queues give you strict ordering and exactly-once processing, but cap throughput and demand a message group ID. The visibility timeout is the invisible clock that starts when a consumer receives a message: get it wrong one way and messages are processed twice, wrong the other way and they are stuck. And the dead-letter queue is the quarantine that stops one poison message from blocking your whole pipeline forever.

By the end you will have created a standard queue and a FIFO queue, sent and received and deleted messages on both, demonstrated a message reappearing because you didn’t delete it in time, wired a dead-letter queue with maxReceiveCount=3, watched a poison message get quarantined, and redriven it back — all with the aws sqs CLI and then again as Terraform, everything free-tier. Then, because your first real queue will misbehave in about a dozen classic ways — messages processed twice, messages stuck invisible, a FIFO queue that silently stops delivering, a DLQ that never catches anything, empty receives quietly running up a bill — you get a symptom-to-fix troubleshooting playbook. Read the prose once; keep the tables open when your own queue misbehaves.

What problem this solves

Direct, synchronous coupling between services fails in production in ways that are hard to see in a demo. When the caller waits for the callee, three things break at once: availability (if the downstream is momentarily down, the request is lost — there is nowhere to hold it), elasticity (a burst of work must be absorbed right now by the downstream, so you size it for the peak and pay for idle the rest of the time), and latency isolation (the caller’s p99 becomes the callee’s p99, so one slow dependency drags the whole chain down). Retries make it worse, not better: the caller retries, the overloaded downstream gets more load, and you have built a thundering herd.

A queue decouples the two sides in time. The producer’s job ends when the message is durably stored; the consumer’s job starts whenever it has capacity. A spike becomes a backlog that drains at the consumer’s sustainable rate instead of an outage. A downstream deploy or crash becomes a few minutes of queued messages instead of lost orders. And because SQS stores each message redundantly across multiple Availability Zones, “durably stored” is a real guarantee, not a single-box promise. This pattern — producer → queue → consumer — is the backbone of nearly every event-driven and microservice architecture on AWS.

What breaks without understanding SQS specifically is subtle, because the service is trivially easy to start and easy to get silently wrong. A beginner sends a message, receives it, processes it, and forgets to delete it — so 30 seconds later it comes back and is processed again, and they conclude “SQS is duplicating my messages” when in fact they never acknowledged the first delivery. Or they pick a standard queue for a payment flow, get a rare duplicate under at-least-once delivery, and double-charge a customer. Or they create a FIFO queue, forget the message group ID, and every send fails. Or they attach a DLQ but point the redrive policy at the wrong ARN, so poison messages pile up invisibly. None of these is an SQS failure — each is a missing piece of the mental model. This article’s whole job is to make those pieces boringly obvious.

Here is the entire field on one screen — the pieces you will meet, what each one is, and the classic beginner trap attached to it:

Learning objectives

By the end of this article you can:

• Explain the producer → queue → consumer decoupling pattern and why polling (SQS) differs from push (SNS) and from streaming (Kinesis).
• Choose correctly between a standard queue (near-unlimited throughput, at-least-once, best-effort order) and a FIFO queue (strict per-group order, exactly-once within the dedup window, capped throughput) — and defend the choice.
• Configure and reason about the visibility timeout — the in-flight window — including why too-short duplicates work and too-long strands it, how to extend it per message with ChangeMessageVisibility, and the ≥ 6× function-timeout rule for Lambda consumers.
• Set message retention (1 minute – 14 days), delivery delay / delay queues, and per-message timers, and know which of these FIFO does not support.
• Turn on long polling (WaitTimeSeconds), understand how it cuts empty receives and cost, and read the SQS CloudWatch metrics that tell you the queue’s health.
• Wire a dead-letter queue with a redrive policy (deadLetterTargetArn + maxReceiveCount), quarantine a poison message, and redrive it back to source once fixed.
• Send message attributes, respect the 256 KB limit (and use S3 / the extended client for large payloads), and use batch send/receive/delete (up to 10) to cut cost 10×.
• Apply encryption (SSE-SQS / SSE-KMS), scope a queue access policy to the exact principal allowed to send, and respect the in-flight limits (120,000 standard / 20,000 FIFO).
• Run a symptom → cause → confirm → fix troubleshooting playbook for the failures that hit every first queue.

Prerequisites & where this fits

You need an AWS account with permission to create SQS queues and (for the Terraform half) IAM (do this in a personal or dev account, never straight into production). You need the AWS CLI v2 installed and configured (aws configure or aws sso login), and Terraform ≥ 1.5 for Part B. Everything here fits comfortably inside the always-free SQS tier (1 million requests per month), so the running cost is effectively zero. You should be comfortable reading JSON and running shell commands; you do not need prior messaging experience.

Where this sits: SQS is the point-to-point buffer at the heart of the AWS integration and event-driven stack. It pairs constantly with two siblings you should read alongside it — Amazon SNS: Pub/Sub Fan-Out, Subscriptions, Filtering & SNS→SQS Patterns for the push, one-to-many half of the pattern (SNS fans a message out; SQS queues each copy for one consumer group), and Event-Driven Architecture on AWS: EventBridge, SQS, SNS, Lambda and Step Functions for how all the integration services fit together. When SQS feeds a Lambda consumer, the failure modes bleed into function land — AWS Lambda Errors, Timeouts & Cold Starts: The Troubleshooting Playbook covers the consumer side. And when message processing itself is the hard part — poison messages, partial-batch failures, idempotency at scale — the advanced sibling SQS Message Processing Failures: Redrive, DLQs, Poison Messages & Idempotency goes deeper than this hands-on introduction.

A quick map of who owns what, so when something misbehaves you look in the right place first:

Core concepts

Six ideas make everything later obvious. Read them once; the deep sections just expand each.

A queue is a durable buffer, not a database and not a stream. You put a message in; a consumer takes it out and removes it. There is no random access, no query, no “read message #4173” — you receive whatever the queue hands you, in roughly the order it can. Unlike a Kinesis or Kafka stream (where records are retained and many consumers replay the same log at their own offsets), an SQS message is normally consumed once by one consumer and then deleted. That single difference — consume-and-delete vs retain-and-replay — is the fastest way to know whether you want SQS or a stream.

SQS is pull, not push. Nothing is delivered to your consumer; your consumer polls — it asks the queue “any messages?” and gets up to ten back. This is the opposite of SNS, which pushes to its subscribers. Pull means the consumer controls its own rate (natural back-pressure) and that an idle consumer must keep asking (which is why long polling matters for cost). When people say “SQS + Lambda,” AWS runs a managed poller on your behalf — but under the hood it is still polling.

Delivery is at-least-once (standard) — plan for duplicates. A standard queue guarantees a message is delivered at least once, which means occasionally more than once. This is not a bug you can configure away; it is a property of a massively distributed system. The practical consequence: your consumer must be idempotent — processing the same message twice must be safe (charge a card once even if you see the event twice, using a dedupe key or a conditional write). FIFO queues raise this to exactly-once within a 5-minute deduplication window, which handles retries but is not a licence to skip idempotency for very late duplicates.

Receiving hides a message; it does not remove it. When you ReceiveMessage, the message is not deleted — it becomes in-flight: invisible to other consumers for the visibility timeout. You are expected to process it and then explicitly DeleteMessage using the receipt handle you got back. If you delete it, it is gone. If you don’t (crash, timeout, forget), the visibility timeout expires and the message reappears for someone else to try. This receive-then-delete handshake is the entire correctness model of SQS, and misunderstanding it is the number-one beginner error.

Every message has a clock ticking on it. Two independent timers govern a message’s life. Retention is how long an undelivered message survives in the queue before SQS discards it (1 minute to 14 days). The visibility timeout is how long a received message stays hidden before redelivery. A message “disappearing” is almost always one of these two expiring — retention (nobody consumed it in time) or a visibility mismatch (it was redelivered while you thought you still owned it).

A dead-letter queue is a safety valve, not an error log. If a message can never be processed — malformed, references a deleted record, triggers a bug — it will otherwise be received, fail, reappear, and be received again forever, blocking the queue behind it (especially on FIFO, where a stuck group halts everything after it). A DLQ with a maxReceiveCount breaks that loop: after N failed receives, SQS moves the message out to a separate queue where you can inspect it, alarm on it, fix the cause, and redrive it back. Every production queue should have one.

The vocabulary in one table

Pin every moving part down before the deep dive. The glossary at the end repeats these for lookup; this is the mental model side by side:

Where SQS sits among the messaging services

Beginners constantly confuse SQS, SNS and Kinesis. One table settles it:

The canonical combination is SNS → SQS fan-out: SNS pushes one event to several SQS queues, each owned by a different consumer group, so every group gets its own durable copy to process at its own pace. That pattern is the SNS sibling’s home turf; here we focus on the queue itself.

Standard vs FIFO: the core decision

This is the decision you make first, at create-queue time, and it is effectively permanent — you cannot convert a standard queue to FIFO or vice versa; you create a new queue and migrate. So understand the trade precisely.

The headline comparison

Read the two delivery rows together, because they are the whole reason FIFO exists. A standard queue is built for scale: it spreads your messages across a huge fleet, which is why it can absorb almost any rate — but that same distribution is why a message can occasionally be delivered twice and why strict global ordering is impossible. A FIFO queue trades that scale for guarantees: messages in the same group come out in exactly the order they went in, and a message sent twice within five minutes is accepted once.

Throughput and limits, side by side

High-throughput FIFO is worth knowing about the moment 300 msg/s feels tight: by scoping deduplication to each message group instead of the whole queue, SQS can process many groups in parallel and lift the ceiling into the thousands (the exact numbers vary by Region — check the current SQS quotas page rather than trusting a memorised figure). The catch is that dedup then only applies within a group, which is usually fine because your group ID is already your natural partition key.

Ordering — what “best-effort” and “strict” really mean

The FIFO “gotcha” hides in the last two rows. Ordering is per group, not per queue — so if you put every message in one group ID you serialise your entire consumer and throttle yourself; if you spread messages across many group IDs (e.g. one per customer, per order, per device) you get parallelism and ordering where it matters. And because a group is strictly ordered, a message that can’t be processed blocks everything behind it in that group until it is deleted or moved to the DLQ — which is exactly why a FIFO queue without a DLQ is dangerous.

FIFO message groups and deduplication

Two IDs govern every FIFO send. Get them straight:

Deduplication comes in two flavours, and choosing wrong causes silent message loss or silent duplicates:

The 5-minute deduplication interval is fixed and often surprises people: if you send the same dedup ID twice 4 minutes apart, the second is swallowed; 6 minutes apart, both go through. So “FIFO didn’t deliver my message” is very often “the dedup ID collided with a recent send,” not a bug.

The decision, distilled

The honest default: reach for a standard queue and make your consumer idempotent unless you have a concrete ordering or no-duplicate requirement — standard is cheaper, faster, and simpler. Move to FIFO when correctness genuinely depends on order or exactly-once, and accept the throughput and group-ID cost that comes with it.

Visibility timeout: the in-flight window

The visibility timeout is the single most misunderstood setting in SQS, and it is the root cause of both “my messages get processed twice” and “my messages are stuck.” Understand it and half of all SQS bugs evaporate.

How it works

When a consumer calls ReceiveMessage, SQS hands back the message and starts a hidden countdown. For the duration of the visibility timeout, that message is in-flight: still in the queue, but invisible to every other ReceiveMessage call. The consumer is expected to finish its work and call DeleteMessage before the timer expires. Two outcomes:

• You delete in time → the message is permanently removed. Done.
• The timer expires first (you crashed, timed out, or forgot) → the message becomes visible again and will be delivered to some consumer on a later receive, with its ApproximateReceiveCount incremented.

The settings

Too short vs too long — the two failure directions

The rule of thumb: set the visibility timeout to a bit more than your consumer’s realistic worst-case processing time — long enough that a healthy consumer always deletes before expiry, short enough that a dead consumer’s message is retried quickly. If your processing time varies wildly, don’t pick a huge fixed timeout; use a modest one and extend the outliers live.

Extending a message you’re still working on

For a long or variable job, don’t over-provision the queue’s visibility timeout — call ChangeMessageVisibility to push this message’s deadline out as you go (a “heartbeat”). This keeps the common case snappy while giving slow messages room.

The ≥ 6× rule for Lambda consumers

When SQS triggers Lambda through an event-source mapping, AWS’s own guidance is to set the queue’s visibility timeout to at least six times the Lambda function’s timeout. The factor accounts for the batching window plus Lambda’s internal retries on the batch, and it prevents a message from becoming visible again (and being picked up by a second concurrent invocation) while the first invocation is still legitimately working on it.

If you ignore this and set the visibility timeout equal to (or below) the function timeout, you get the classic SQS-to-Lambda double-processing bug: the message reappears mid-invocation and a second Lambda picks it up, doing the work twice. The Lambda troubleshooting playbook covers the consumer-side symptoms in depth.

Message lifecycle: retention, delay and timers

Beyond the visibility timeout, two more timers shape a message’s life: how long it can wait to be read, and how long before it becomes readable.

Retention — how long an unread message lives

Retention is your backstop against a down consumer: with the 4-day default, your workers can be offline for the weekend and no messages are lost. But it is also a silent killer — if a message is never consumed (a poison message with no DLQ, or a consumer that’s been down longer than the retention period), SQS discards it with no notification. “Where did my message go?” with no DLQ and a long-down consumer is almost always retention expiry. Set retention to cover your worst realistic outage, and add a DLQ so poison messages are quarantined before they age out.

Delivery delay and message timers

Sometimes you want a message not to be available immediately — to defer a reminder, throttle a retry, or schedule work. Two mechanisms:

Two things to remember. First, the maximum delay is 15 minutes — SQS is not a general-purpose scheduler; for longer or calendar-based delays use EventBridge Scheduler or Step Functions Wait. Second, FIFO queues do not support per-message timers — you can set a queue-level delay on a FIFO queue, but you cannot delay individual messages; attempting per-message delay on FIFO is simply ignored.

The three timers, disambiguated

Because “how long” appears three times in SQS, beginners conflate them. Keep them separate:

Polling: long vs short

Because SQS is pull-based, how your consumer asks for messages has a real effect on both latency and cost.

The comparison

Short polling is a historical default that quietly costs money: an idle consumer looping on short polls issues thousands of ReceiveMessage calls that each return nothing — and empty receives are billable requests. Long polling collapses that into a single call that simply waits, so an idle queue costs almost nothing and a message is returned the instant it arrives.

Turning it on

Set ReceiveMessageWaitTimeSeconds=20 on the queue and you never think about it again. The only reason to use a lower value is a latency-critical path that can’t tolerate up to a 20-second wait on an empty queue — rare in practice, since a message that is there is returned immediately regardless.

The receive request, fully enumerated

Every ReceiveMessage has a handful of parameters worth knowing:

Always request MaxNumberOfMessages=10 and MessageSystemAttributeNames=ApproximateReceiveCount — the first cuts your request count by up to 10×, and the second lets your consumer see how many times a message has already failed (so you can log or side-line it before the DLQ even kicks in).

Dead-letter queues and redrive

A dead-letter queue is an ordinary SQS queue that you designate as the destination for messages a source queue can’t process. It is wired up not on the DLQ itself but via a redrive policy on the source queue.

The redrive policy

The mechanic is exact: every time a message is received and not deleted (because processing failed and the visibility timeout lapsed), its ApproximateReceiveCount increments. When that count exceeds maxReceiveCount, the next receive attempt sends the message to the DLQ instead of to your consumer. So maxReceiveCount=3 means “give it three honest tries, then quarantine it.”

Type-matching and the redrive allow policy

The redrive allow policy sits on the DLQ and restricts which source queues are allowed to target it — allowAll (default), denyAll, or byQueue with an explicit list of source ARNs. In production you set byQueue so a stray queue can’t dump into your DLQ.

Redrive back to source

A DLQ is a holding pen, not a graveyard. Once you’ve found and fixed the cause (a bug, a since-restored downstream), you redrive the quarantined messages back to the original queue for reprocessing.

The console exposes this as the “Start DLQ redrive” button; the CLI/API call is StartMessageMoveTask. Redrive to source is the payoff of having a DLQ: nothing is lost, you just replay after the fix.

Message anatomy: attributes, size, batching and large payloads

The message and its attributes

Message attributes let a consumer route or filter without parsing the body — for example an eventType attribute so a worker can continue past messages it doesn’t handle. But remember they count toward the 256 KB ceiling, so they’re for small metadata, not a second payload.

The 256 KB limit and large payloads

For anything over 256 KB the idiomatic answer is the SQS Extended Client Library (a wrapper, originally for Java, now available for other SDKs) that transparently offloads the body to S3 and passes only a lightweight pointer through the queue — messages up to 2 GB. You can also do this by hand: PutObject to S3, send the object key as the message, and have the consumer GetObject. Either way, the messaging plane stays small and cheap while the bytes live in S3.

Batch operations — the 10× cost lever

Batching is the biggest cost lever in SQS: because you’re billed per request, moving from one-message-at-a-time to batches of 10 cuts your request bill by up to 90%. The catch on send is that the combined batch payload must still fit in 256 KB, and each batch entry needs a distinct Id (duplicate IDs → BatchEntryIdsNotDistinct). On a partial batch failure, the API returns per-entry Successful/Failed lists — you must inspect them and retry only the failures.

Encryption, access policy and limits

Encryption

SSE-SQS is free and on by default for new queues, so there is no reason to run an unencrypted queue. Reach for SSE-KMS only when you need a customer-managed key for audit or cross-account key policies — and then grant the producer/consumer principals kms:GenerateDataKey and kms:Decrypt on the key, or sends/receives fail with a KMS error.

Access policy — who may send and receive

SQS uses a resource-based policy (like an S3 bucket policy) on the queue. It’s how you let another service or account put messages in without sharing credentials.

The critical row is the condition: when you let sns.amazonaws.com or s3.amazonaws.com send to your queue, always pin aws:SourceArn (and ideally aws:SourceAccount) to the exact topic or bucket. Without it, any SNS topic or S3 bucket in any account could be configured to dump into your queue — the classic confused-deputy hole.

Limits and quotas you’ll actually hit

The core API, at a glance

Architecture at a glance

The diagram below is the exact shape you build in the lab, drawn as a message path you can follow left to right. A producer (your app, or an SNS topic / S3 event fanning work in) calls SendMessage — batching up to ten at a time, and, for a FIFO queue, attaching a message group ID and a deduplication ID. The message lands in the SQS queue (standard for throughput, or a .fifo queue for order and exactly-once). A consumer long-polls a batch; each received message goes in-flight, hidden for the visibility timeout while the consumer works. On success the consumer calls DeleteMessage and the message is gone; on failure it stays hidden until the timeout lapses, then reappears for another try. Once a message has been received more times than maxReceiveCount, SQS moves it out to the dead-letter queue, where you inspect, fix, and redrive it back.

The six numbered badges mark the six places this pipeline breaks if you misconfigure it, and the legend narrates each as concept · symptom · fix — the same map as the troubleshooting playbook, drawn onto the architecture so you can see where each failure lives.

Real-world scenario

KloudCart, a mid-size online grocer in Pune, ran its checkout on a synchronous chain: the order API called an inventory service, which called a payment service, which called a fulfilment service, all in one request. It worked in testing and fell apart on their first big festival sale. When payment slowed under load, the whole checkout slowed; when fulfilment briefly restarted during a deploy, in-flight orders were lost with no record. Their SRE, Meera, was asked to “make checkout resilient,” and SQS was the core of her fix — along with every classic beginner mistake on the way.

She started by decoupling fulfilment behind a standard queue: the order API now just SendMessages an order event and returns instantly, and a fleet of fulfilment workers polls the queue. Checkout latency dropped immediately and a fulfilment restart became a harmless few-minutes backlog. But within a day, support reported duplicate shipments — a handful of customers got two of the same order. The cause was two-fold: standard queues are at-least-once (so occasional duplicates are expected), and her workers’ visibility timeout was 30 seconds while some fulfilment calls took 45, so a slow message reappeared mid-processing and a second worker shipped it too (playbook rows 1 and 2). She fixed both: she raised the visibility timeout to 120 s (comfortably above the p99), added a ChangeMessageVisibility heartbeat for the rare very-slow order, and — the real fix — made the worker idempotent by recording each processed orderId and skipping duplicates. Duplicates stopped.

Next, one malformed order (a corrupt line-item from an upstream bug) began looping: received, failed, reappeared, failed, forever — and because it kept occupying a worker, throughput sagged. She had no DLQ. She created one and set a redrive policy with maxReceiveCount=3; the poison order was quarantined after three tries, a CloudWatch alarm on the DLQ’s ApproximateNumberOfMessagesVisible paged her, she found and fixed the upstream bug, and redrove the message back to the source queue to reprocess cleanly (rows 9, 10). Nothing was lost.

The last problem was the ledger: KloudCart’s finance events (charge, refund, adjustment) for a given customer had to be processed in order and never twice — a standard queue couldn’t promise either. For that one stream she moved to a FIFO queue with MessageGroupId = customerId, so each customer’s events were strictly ordered while different customers ran in parallel, and content-based deduplication caught accidental double-sends within the 5-minute window. Her first FIFO attempt failed every send with MissingParameter because she forgot the group ID (row 5) — a two-line fix once she understood that FIFO requires it. The result: checkout p99 fell from 3.1 s to 240 ms, festival-day order loss went to zero, duplicate shipments stopped, and the finance ledger became provably ordered and exactly-once. Meera’s wiki note was the thesis of this article: “SQS never dropped or duplicated anything I didn’t ask it to. Every bug was me — no delete, a short visibility timeout, no DLQ, or a missing FIFO key.”

Advantages and disadvantages

When each side matters: SQS’s advantages dominate for decoupling and buffering work between services — absorbing spikes, surviving downstream outages, smoothing bursty load into a steady drain. The disadvantages bite when you need fan-out (one message to many consumer groups — use SNS→SQS), replay / multiple independent readers of the same log (use Kinesis or Kafka), very large payloads (offload to S3), or strict global ordering (FIFO only orders within a group). The honest rule: use SQS whenever one producer needs to hand work to one pool of consumers reliably; layer SNS in front for fan-out and reach for a stream when you need retained, replayable history.

Hands-on lab

You will build the diagram: a standard queue and a FIFO queue, send/receive/delete on both, demonstrate visibility-timeout redelivery, then wire a DLQ with maxReceiveCount=3, quarantine a poison message, and redrive it — first with the aws sqs CLI, then the identical thing as Terraform. Everything is free-tier. Pick a region and stick to it (this lab uses ap-south-1, Mumbai).

⚠️ Cost note: SQS gives you 1,000,000 requests free every month, so this whole lab costs effectively ₹0. The only way to accrue anything is to leave a busy short-polling loop running; we delete every queue at the end.

What you’ll create

Part A — the CLI path

Step 1 — Create a standard queue. The simplest possible queue:

aws sqs create-queue \
  --queue-name kv-orders \
  --attributes VisibilityTimeout=30,MessageRetentionPeriod=345600,ReceiveMessageWaitTimeSeconds=20 \
  --region ap-south-1

Expected — the queue URL you’ll use for everything else:

{ "QueueUrl": "https://sqs.ap-south-1.amazonaws.com/111122223333/kv-orders" }

Capture it in a variable:

STD_URL=$(aws sqs get-queue-url --queue-name kv-orders --region ap-south-1 --query QueueUrl --output text)

Step 2 — Create a FIFO queue. The name must end in .fifo, and we turn on content-based deduplication so we don’t have to pass a dedup ID on every send:

aws sqs create-queue \
  --queue-name kv-orders.fifo \
  --attributes FifoQueue=true,ContentBasedDeduplication=true,VisibilityTimeout=30 \
  --region ap-south-1

FIFO_URL=$(aws sqs get-queue-url --queue-name kv-orders.fifo --region ap-south-1 --query QueueUrl --output text)

Expected: a QueueUrl ending kv-orders.fifo. Omit the .fifo suffix and you get InvalidParameterValue: The name of a FIFO queue can only include ... and must end with the .fifo suffix.

Step 3 — Send messages. First to the standard queue:

aws sqs send-message \
  --queue-url "$STD_URL" \
  --message-body '{"orderId":"1001","item":"apples"}' \
  --region ap-south-1

Expected: a MessageId and an MD5OfMessageBody. Now the FIFO queue — note the required --message-group-id:

aws sqs send-message \
  --queue-url "$FIFO_URL" \
  --message-body '{"orderId":"2001","event":"charge"}' \
  --message-group-id "customer-42" \
  --region ap-south-1

Try it without --message-group-id to see the guardrail:

aws sqs send-message --queue-url "$FIFO_URL" --message-body '{"x":1}' --region ap-south-1
# An error occurred (MissingParameter): The request must contain the parameter MessageGroupId.

Step 4 — Receive and delete (the acknowledge handshake). Long-poll a batch and ask for the receive count:

aws sqs receive-message \
  --queue-url "$STD_URL" \
  --max-number-of-messages 10 \
  --wait-time-seconds 20 \
  --message-system-attribute-names ApproximateReceiveCount \
  --region ap-south-1

Expected — one message with a ReceiptHandle (a long opaque token) and "ApproximateReceiveCount": "1". Copy the receipt handle and delete the message to acknowledge it:

RH="AQEB...the-long-receipt-handle..."
aws sqs delete-message --queue-url "$STD_URL" --receipt-handle "$RH" --region ap-south-1

The delete returns nothing (success). The message is now permanently gone — receive again and the queue is empty.

Step 5 — Demonstrate visibility-timeout redelivery. This is the core lesson. Send a message, receive it (which hides it), don’t delete it, and watch it come back. First shorten the visibility timeout so you don’t have to wait 30 s:

aws sqs send-message --queue-url "$STD_URL" \
  --message-body '{"orderId":"1002"}' --region ap-south-1

# Receive with a SHORT 5-second visibility timeout, and do NOT delete:
aws sqs receive-message --queue-url "$STD_URL" \
  --visibility-timeout 5 \
  --message-system-attribute-names ApproximateReceiveCount \
  --region ap-south-1
# → ApproximateReceiveCount: "1"

Wait ~6 seconds (past the 5-second timeout), then receive again:

sleep 6
aws sqs receive-message --queue-url "$STD_URL" \
  --message-system-attribute-names ApproximateReceiveCount \
  --region ap-south-1
# → the SAME message, now ApproximateReceiveCount: "2"

There it is: you never deleted it, so it reappeared — with the receive count incremented. This is exactly the mechanism behind “SQS processed my message twice” (a too-short timeout) and the mechanism a DLQ uses to count failures. Delete it now (fetch a fresh receipt handle from the last receive — receipt handles change on every receive):

RH2="...fresh-handle-from-the-second-receive..."
aws sqs delete-message --queue-url "$STD_URL" --receipt-handle "$RH2" --region ap-south-1

Step 6 — Create a DLQ and attach a redrive policy. Create the dead-letter queue and grab its ARN:

aws sqs create-queue --queue-name kv-orders-dlq --region ap-south-1
DLQ_URL=$(aws sqs get-queue-url --queue-name kv-orders-dlq --region ap-south-1 --query QueueUrl --output text)
DLQ_ARN=$(aws sqs get-queue-attributes --queue-url "$DLQ_URL" \
  --attribute-names QueueArn --region ap-south-1 --query Attributes.QueueArn --output text)
echo "$DLQ_ARN"
# arn:aws:sqs:ap-south-1:111122223333:kv-orders-dlq

The redrive policy value is itself a JSON string, which is fiddly to escape on the command line — write it to a file:

cat > redrive.json <<JSON
{ "RedrivePolicy": "{\"deadLetterTargetArn\":\"${DLQ_ARN}\",\"maxReceiveCount\":\"3\"}" }
JSON

aws sqs set-queue-attributes --queue-url "$STD_URL" \
  --attributes file://redrive.json --region ap-south-1

Verify it stuck:

aws sqs get-queue-attributes --queue-url "$STD_URL" \
  --attribute-names RedrivePolicy --region ap-south-1
# → "RedrivePolicy": "{\"deadLetterTargetArn\":\"...kv-orders-dlq\",\"maxReceiveCount\":\"3\"}"

Step 7 — Quarantine a poison message. Send a “poison” message, then receive-without-delete four times (each with a short visibility timeout so it reappears fast). After the 3rd failed receive, the 4th attempt moves it to the DLQ instead of returning it:

aws sqs send-message --queue-url "$STD_URL" \
  --message-body '{"orderId":"BAD","corrupt":true}' --region ap-south-1

for i in 1 2 3 4; do
  echo "--- receive attempt $i ---"
  aws sqs receive-message --queue-url "$STD_URL" \
    --visibility-timeout 1 \
    --message-system-attribute-names ApproximateReceiveCount \
    --region ap-south-1 --query 'Messages[0].Attributes.ApproximateReceiveCount' --output text
  sleep 2
done

Expected: attempts 1–3 print 1, 2, 3 (the message keeps coming back); by attempt 4 the source queue returns nothing because SQS has moved the message to the DLQ. Confirm it’s in the DLQ:

aws sqs receive-message --queue-url "$DLQ_URL" \
  --message-system-attribute-names ApproximateReceiveCount \
  --region ap-south-1 --query 'Messages[0].Body' --output text
# {"orderId":"BAD","corrupt":true}

Step 8 — Redrive it back to source. Once you’d fixed the root cause, you’d replay the quarantined messages. Start a message-move task from the DLQ back to its source:

aws sqs start-message-move-task --source-arn "$DLQ_ARN" --region ap-south-1
# → { "TaskHandle": "..." }

aws sqs list-message-move-tasks --source-arn "$DLQ_ARN" --region ap-south-1
# → Status: COMPLETED, ApproximateNumberOfMessagesMoved: 1

The message is now back in kv-orders, ready to be processed cleanly.

Step 9 — Verify and tear down. Delete all three queues:

aws sqs delete-queue --queue-url "$STD_URL"  --region ap-south-1
aws sqs delete-queue --queue-url "$FIFO_URL" --region ap-south-1
aws sqs delete-queue --queue-url "$DLQ_URL"  --region ap-south-1

Part B — the same thing as Terraform

The CLI is great for learning; Terraform is how you keep it. This main.tf reproduces the whole lab declaratively — both queue types, the DLQ, the redrive policy, a redrive allow policy, and an access policy showing who may send:

terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 5.0" }
  }
}

provider "aws" {
  region = "ap-south-1"
}

# 1. The dead-letter queue (created first so the source can reference its ARN)
resource "aws_sqs_queue" "dlq" {
  name                      = "kv-orders-dlq"
  message_retention_seconds = 1209600 # 14 days — keep quarantined messages long enough to inspect
  sqs_managed_sse_enabled   = true
}

# 2. The main standard queue, wired to the DLQ
resource "aws_sqs_queue" "orders" {
  name                       = "kv-orders"
  visibility_timeout_seconds = 120    # comfortably above worst-case processing time
  message_retention_seconds  = 345600 # 4 days
  receive_wait_time_seconds  = 20     # long polling by default
  max_message_size           = 262144 # 256 KB
  delay_seconds              = 0
  sqs_managed_sse_enabled    = true

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dlq.arn
    maxReceiveCount     = 3
  })
}

# 3. Restrict which queues may use the DLQ (redrive ALLOW policy sits on the DLQ)
resource "aws_sqs_queue_redrive_allow_policy" "dlq_allow" {
  queue_url = aws_sqs_queue.dlq.id
  redrive_allow_policy = jsonencode({
    redrivePermission = "byQueue"
    sourceQueueArns   = [aws_sqs_queue.orders.arn]
  })
}

# 4. A FIFO queue with content-based dedup and high-throughput mode
resource "aws_sqs_queue" "orders_fifo" {
  name                        = "kv-orders.fifo"
  fifo_queue                  = true
  content_based_deduplication = true
  deduplication_scope         = "messageGroup"     # required for high throughput
  fifo_throughput_limit       = "perMessageGroupId"
  visibility_timeout_seconds  = 30
  sqs_managed_sse_enabled     = true
}

# 5. Access policy: allow ONLY a specific SNS topic to send (confused-deputy guard)
resource "aws_sqs_queue_policy" "orders_allow_sns" {
  queue_url = aws_sqs_queue.orders.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "AllowOneTopic"
      Effect    = "Allow"
      Principal = { Service = "sns.amazonaws.com" }
      Action    = "sqs:SendMessage"
      Resource  = aws_sqs_queue.orders.arn
      Condition = {
        ArnEquals = { "aws:SourceArn" = "arn:aws:sns:ap-south-1:111122223333:kv-topic" }
      }
    }]
  })
}

output "std_queue_url"  { value = aws_sqs_queue.orders.url }
output "fifo_queue_url" { value = aws_sqs_queue.orders_fifo.url }
output "dlq_arn"        { value = aws_sqs_queue.dlq.arn }

terraform init
terraform apply    # review the plan, type yes
aws sqs send-message --queue-url "$(terraform output -raw std_queue_url)" \
  --message-body '{"orderId":"3001"}' --region ap-south-1
terraform destroy  # clean teardown, one command

Common mistakes & troubleshooting

This is the section you’ll return to. It is a playbook: match your symptom, run the confirm command, apply the fix. Rows tagged (pointer) are covered in depth by the advanced processing sibling.

The SQS error / status reference

When an API call fails, the error code tells you exactly what to fix:

The CloudWatch metrics that tell you the truth

You can’t fix what you can’t see. These are the SQS metrics to graph and alarm on (default 5-minute granularity):

The two alarms every production queue needs: ApproximateAgeOfOldestMessage on the main queue (catches a stalled or starved consumer) and ApproximateNumberOfMessagesVisible on the DLQ > 0 (catches poison messages the moment they’re quarantined).

The three nastiest, explained

“SQS is duplicating my messages” (rows 1, 2) is almost never SQS’s fault. Two distinct things get blamed as “duplication.” The first is real at-least-once delivery on a standard queue — rare, expected, and handled by making your consumer idempotent (record a business key, use a conditional write). The second — far more common for beginners — is a visibility-timeout mismatch: your processing takes longer than the timeout, so the message reappears and a second consumer grabs it while the first is still working. That’s not SQS duplicating; that’s you telling SQS “I’m done owning this” (by letting the timeout lapse) before you actually were. Confirm by watching ApproximateReceiveCount climb during a single logical unit of work; fix by raising the timeout above your p99 and heart-beating the outliers with ChangeMessageVisibility.

“My DLQ never catches anything” (row 9) usually means the redrive policy is subtly wrong. The three failure modes: the policy isn’t set at all (so failures loop forever until retention); the deadLetterTargetArn points at the wrong queue (a typo, or a queue in another Region/account); or there’s a type mismatch — a standard source pointing at a FIFO DLQ, or vice versa, which SQS rejects. Confirm with get-queue-attributes --attribute-names RedrivePolicy and eyeball the ARN and the FIFO/standard match. And remember the count semantics: maxReceiveCount=3 means the message is moved on the receive after the third failed one — so with a very long visibility timeout it can take a long wall-clock time to reach the DLQ even though only three receives happened.

FIFO “stops delivering” (rows 6, 7) trips up everyone new to FIFO, and it’s two different traps wearing the same coat. Trap one is deduplication swallowing sends: because FIFO dedupes on a 5-minute window, re-sending the same content (content-based dedup) or the same MessageDeduplicationId inside that window is accepted but silently not delivered — it looks like the message vanished. Trap two is a blocked group: FIFO guarantees order within a group, so a message that can’t be processed blocks every message behind it in the same group until it’s deleted or moved to the DLQ. If everything is in one group ID, one poison message halts the entire queue. The fixes: use genuinely unique dedup IDs per logical message, spread work across many group IDs (one per entity), and always attach a FIFO DLQ so a poison message is evicted instead of blocking its group forever. The advanced processing sibling goes deep on poison-message handling and partial-batch failures.

Best practices

• Always delete after real success — never before. The delete is your acknowledgement. Delete only when the work is genuinely done, and design so a redelivery (which will happen) is harmless.
• Make consumers idempotent from day one. At-least-once delivery is a property, not a bug. Dedupe on a business key or use conditional writes so processing the same message twice is safe — this single habit prevents most SQS incidents.
• Set the visibility timeout to a bit above your p99 processing time, and heartbeat outliers with ChangeMessageVisibility rather than setting one enormous fixed timeout. For Lambda consumers, use ≥ 6× the function timeout.
• Give every production queue a DLQ with maxReceiveCount 3–5, and alarm on the DLQ’s depth. A queue without a DLQ lets poison messages loop or silently age out.
• Turn on long polling everywhere (ReceiveMessageWaitTimeSeconds=20). It cuts empty receives, latency and cost with no downside for virtually all workloads.
• Batch send, receive and delete (up to 10) wherever you can — it’s the biggest cost lever, cutting your request bill by up to 90%.
• Prefer standard + idempotency; reach for FIFO only when order or exactly-once is a real requirement. FIFO’s throughput cap and group-ID discipline are a real cost.
• On FIFO, choose the group ID as your natural partition key (customer, order, device) so you get ordering where it matters and parallelism across entities — never one global group.
• Keep messages small; offload large payloads to S3 via the extended client or a pointer. The messaging plane should carry references, not megabytes.
• Encrypt at rest (SSE-SQS is free and default) and scope the access policy with aws:SourceArn so only the intended topic/bucket/service can send — closing the confused-deputy hole.
• Set a retention period that covers your worst realistic consumer outage (often the 14-day max for critical queues), so a long downstream outage doesn’t quietly discard work.
• Right-size and monitor: alarm on ApproximateAgeOfOldestMessage (stalled consumer) and DLQ depth (poison messages), and graph backlog vs. delete rate to spot a consumer falling behind.

Security notes

SQS’s security model is small but load-bearing:

The two you’ll get wrong first: a queue policy that grants sqs:* to Principal: "*" “just to get it working” (which lets anyone in the account — or worse — send and purge), and forgetting the aws:SourceArn condition when SNS or S3 sends to the queue (leaving it open to any source). Scope both tightly from the start; loosening later is easy, tightening after an incident is not.

Cost & sizing

SQS pricing is refreshingly simple — you pay per request, with a large always-free tier and no per-hour or storage charge for the queue itself.

A worked example to make it concrete:

The two takeaways: batching and long polling are the whole cost story — an unbatched, short-polling consumer can cost 10–100× a batched, long-polling one for the same real work — and payload size matters because each 64 KB chunk is a separate billable request. For sizing, watch backlog depth vs. delete rate: if ApproximateNumberOfMessagesVisible trends up, add consumers (SQS scales infinitely on the queue side; the bottleneck is always your consumer fleet).

Interview & exam questions

1. What is Amazon SQS in one sentence, and what problem does it solve? SQS is a fully managed, durable message queue that decouples producers from consumers in time: the producer sends and moves on, the queue buffers the work across multiple AZs, and the consumer polls and processes at its own pace — turning spikes into backlogs and downstream outages into harmless delays. (CLF-C02)

2. Standard vs FIFO — the core differences? Standard: near-unlimited throughput, at-least-once delivery (duplicates possible), best-effort ordering. FIFO: strict ordering within a message group, exactly-once processing within a 5-minute dedup window, capped at 300 msg/s (3,000 batched, more in high-throughput mode), name must end .fifo, and every send needs a MessageGroupId. (DVA-C02, SAA-C03)

3. Explain the visibility timeout and its two failure directions. When a consumer receives a message it becomes in-flight (invisible) for the visibility timeout; the consumer must delete it before the timer expires or it’s redelivered. Too short → the message reappears mid-processing and is handled twice; too long → a crashed consumer’s message is stuck invisible for ages. Set it above p99 processing time and extend outliers with ChangeMessageVisibility. (DVA-C02)

4. Why must SQS consumers be idempotent? Standard queues are at-least-once, so a message can be delivered more than once (by design, and via visibility-timeout redelivery). Processing must be safe to repeat — via a dedupe key or conditional write — so a duplicate doesn’t double-charge or double-ship. (DVA-C02, SAA-C03)

5. How does a dead-letter queue work? A redrive policy on the source queue names a deadLetterTargetArn and a maxReceiveCount; after a message is received that many times without being deleted, SQS moves it to the DLQ. This quarantines poison messages so they stop looping (or blocking a FIFO group), lets you alarm and inspect, and you can redrive them back to source after fixing the cause. (DVA-C02, SAA-C03)

6. Long polling vs short polling — which and why? Long polling (WaitTimeSeconds 1–20) waits for a message across all servers, cutting empty receives, latency and cost; short polling (0) samples a subset and returns immediately, often empty, and each empty receive is a billable request. Use long polling almost always. (DVA-C02)

7. What’s the maximum message size and how do you exceed it? 256 KB (body + attributes). For larger payloads use the SQS Extended Client Library (or a manual S3 pointer), which stores the body in S3 (up to 2 GB) and passes only a reference through the queue. (DVA-C02)

8. On a FIFO queue, what do MessageGroupId and MessageDeduplicationId do? MessageGroupId is the ordering + parallelism unit — messages in the same group are strictly ordered; different groups run in parallel. MessageDeduplicationId (or content-based dedup) is the exactly-once key: a repeat within 5 minutes is accepted but not delivered. (DVA-C02, SAA-C03)

9. Your SQS-triggered Lambda processes some messages twice. First hypothesis? The queue’s visibility timeout is too low relative to the function timeout — set it to at least 6× the Lambda timeout so a message doesn’t reappear (and get picked up by a second invocation) while the first is still processing. Also verify the consumer deletes on success and is idempotent. (DVA-C02)

10. How is SQS priced, and what’s always free? Per request: 1M requests/month free forever, then $0.40/M (standard) or $0.50/M (FIFO); each 64 KB of payload is a request, and empty receives count. Batching (10×) and long polling are the main cost levers. (CLF-C02)

11. When would you choose SQS over SNS or Kinesis? SQS when one consumer group processes each message with durable buffering and back-pressure. SNS when you need to fan one message out to many subscribers (push). Kinesis/Kafka when you need a retained, replayable log with multiple independent readers at their own offsets. The classic combo is SNS→SQS for durable fan-out. (SAA-C03)

12. A message disappeared from a queue with no DLQ. What happened? Almost certainly retention expiry — an undelivered message is discarded after MessageRetentionPeriod (default 4 days, max 14) with no notification. Either the consumer was down longer than retention or it was a poison message looping until it aged out. Add a DLQ and size retention to cover outages. (DVA-C02)

Quick check

1. You receive a message, process it successfully, but forget to call DeleteMessage. What happens ~30 seconds later, and why?
2. Your consumer’s processing takes 45 seconds but the visibility timeout is 30 seconds. What bug appears, and what are two fixes?
3. On a FIFO queue, what two things must a send include (given content-based dedup is off), and what happens if you omit the group ID?
4. You attach a DLQ with maxReceiveCount=3. After how many failed receives is a message moved to the DLQ, and how do you get it back?
5. An idle consumer is running up your SQS bill. What is it almost certainly doing, and what’s the one-line fix?

Answers

1. The message becomes visible again and is redelivered — because receiving only hides a message for the visibility timeout; the explicit DeleteMessage is the acknowledgement, and without it SQS assumes processing failed and re-queues it (with ApproximateReceiveCount incremented).
2. Double processing — the message reappears at 30 s while the first consumer is still working, so a second consumer grabs it. Fixes: raise the visibility timeout above the real processing time (e.g. 90–120 s), and/or call ChangeMessageVisibility to heartbeat the message while working; and make the consumer idempotent as a backstop.
3. A MessageGroupId (the ordering unit) and a MessageDeduplicationId (the exactly-once key). Omit the group ID and the send is rejected with MissingParameter — FIFO requires it on every send.
4. After the 3rd failed receive, the next receive attempt moves it to the DLQ (it’s moved when the count exceeds maxReceiveCount). Get it back by fixing the root cause and running start-message-move-task (the console’s “DLQ redrive”) to move messages back to the source queue.
5. Short polling — looping on ReceiveMessage with WaitTimeSeconds=0, so it issues constant empty receives that are each billable. One-line fix: set ReceiveMessageWaitTimeSeconds=20 on the queue (long polling).

Glossary

Next steps

• Add fan-out in front of the queue. Learn the push, one-to-many half of the pattern — and the canonical SNS→SQS durable fan-out — in Amazon SNS: Pub/Sub Fan-Out, Subscriptions, Filtering & SNS→SQS Patterns.
• Go deep on message-processing failures. Poison messages, partial-batch failures, redrive strategy and idempotency at scale are the subject of SQS Message Processing Failures: Redrive, DLQs, Poison Messages & Idempotency.
• See the whole integration picture. How SQS, SNS, EventBridge, Lambda and Step Functions fit together is laid out in Event-Driven Architecture on AWS: EventBridge, SQS, SNS, Lambda and Step Functions.
• Handle the consumer side. When a Lambda consumes your queue, its errors, timeouts and cold starts matter — work through AWS Lambda Errors, Timeouts & Cold Starts: The Troubleshooting Playbook.