GCP Pub/Sub and Event-Driven Architecture: Decouple and Scale

Quick take: Cloud Pub/Sub is the nervous system of event-driven GCP. It decouples producers from consumers, buffers bursts, and lets many services react to the same event without knowing about each other — but only if you get the subscription type, the ack deadline, the ordering model and the dead-letter policy right.

At 11:40 on a sale morning, a retail platform’s checkout service was calling four downstream services synchronously — inventory, payments, email, analytics — one HTTP call after another, inside the request that the customer was waiting on. The analytics endpoint hit a slow query and started taking eight seconds. Because checkout awaited it, checkout requests piled up, the thread pool drained, and the whole storefront returned 500s. Analytics — the least important consumer of an order — took down the most important producer. That is the failure mode synchronous chains are built to have: the slowest, least-critical dependency sets the latency and the availability of the entire flow.

The fix was Cloud Pub/Sub — Google’s fully managed, global, asynchronous messaging service. Checkout now publishes one order-placed message to a topic and returns in single-digit milliseconds. Inventory, payments, email and analytics each own a subscription and consume at their own pace. When analytics lags, its subscription backlog grows — a number on a dashboard — while checkout, payments and email are completely unaffected. Pub/Sub absorbed a flash-sale burst of 30,000 messages/second without a capacity ticket, because throughput provisioning is Google’s problem, not yours. This is the core promise: decouple producers from consumers in time, in load, and in failure.

But “just put a queue in front of it” is where most teams stop and most production incidents begin. Pub/Sub is not one thing — it is a topic plus a family of subscription types (pull, streaming pull, push, BigQuery, Cloud Storage), each with different delivery semantics, different failure modes and a different bill. Getting it right means understanding the at-least-once default and what duplicates do to your handlers, when exactly-once delivery is worth its constraints, how ordering keys trade throughput for order, why your ack deadline is the single most misconfigured setting in the system, and how dead-letter topics and retry policy keep one poison message from wedging a subscription forever. This article is the production playbook for all of it, with real gcloud, real client code, and a table for every decision.

What problem this solves

Synchronous request-response chains are brittle in three independent ways, and Pub/Sub addresses all three. Temporal coupling: the caller and callee must both be up at the same instant; if the callee is restarting, deploying, or briefly overloaded, the caller fails too. Load coupling: a burst that the producer can absorb (it just accepts HTTP and returns) is forced immediately onto every downstream, so the slowest consumer’s capacity becomes the system’s capacity. Failure coupling: an error or timeout in any link propagates back up the chain, so a non-critical leaf (analytics, a recommendation refresh, a webhook to a partner) can fail a critical root (checkout, payment capture).

Pub/Sub breaks all three by inserting a durable, elastic buffer between producer and consumer. The producer publishes and is done — it does not know or care who is listening, how many subscribers there are, or whether they are healthy. Each consumer reads from its own subscription, which holds an independent backlog and retries independently. A slow consumer grows its backlog (visible, alertable, recoverable) instead of pushing back on the producer. A down consumer simply resumes from where it left off when it recovers, within the message retention window (default 7 days, up to 31). Add a fifth consumer next quarter? Create a subscription; nobody who already publishes or consumes changes a line of code. That is fan-out, and it is the feature that turns a tangle of point-to-point calls into a clean event mesh.

Who hits the pain this solves: anyone whose checkout/ingest/upload path fans out to multiple downstreams; anyone whose traffic is bursty (sales, batch loads, IoT device storms, log spikes); anyone integrating systems that deploy and fail on independent schedules (microservices, partner webhooks, cross-team pipelines); and anyone streaming events into analytics, where you want the operational path fast and the analytical path eventually-consistent. The catch — and this article exists for it — is that Pub/Sub trades synchronous simplicity for eventual consistency, at-least-once duplicates, and a set of delivery knobs you must understand. Misuse any of them and you get duplicate charges, out-of-order state, silently stuck subscriptions, or a telemetry bill that dwarfs your compute. The point of what follows is to use the trade deliberately.

Learning objectives

By the end of this article you can:

• Model an event-driven system on Pub/Sub: choose topic boundaries, design subscriptions per consumer, and reason about fan-out, fan-in and replay.
• Choose the right subscription type — pull, streaming pull, push, BigQuery, or Cloud Storage — for each consumer, and explain the delivery, scaling and cost trade-offs of each.
• Configure the delivery contract correctly: ack deadline, modifyAckDeadline lease extension, at-least-once vs exactly-once, message ordering with ordering keys, and flow control.
• Build a resilient failure path with a dead-letter topic, an exponential retry policy, and idempotent, retry-safe handlers — and know exactly what each setting does to redelivery.
• Use schemas (Avro / Protocol Buffers) to enforce a message contract at publish time, and evolve them without breaking subscribers.
• Decide between Pub/Sub and Pub/Sub Lite on cost, scale, ordering and operational model — and know when each is the wrong choice.
• Wire Pub/Sub to the rest of GCP — Cloud Run / Cloud Functions (push), Dataflow (streaming pipelines), BigQuery and Cloud Storage (native subscriptions), and Eventarc — and grant least-privilege IAM for publish, subscribe and the service agents involved.
• Operate it in production: monitor backlog and oldest-unacked-age, alert on the right signals, control cost, and run a real hands-on lab end to end.

Prerequisites & where this fits

You should be comfortable with a GCP project and the gcloud CLI (auth, setting a project, reading JSON/table output in Cloud Shell), and with basic IAM — that access is granted by binding a role to a principal on a resource. The Google Cloud IAM Explained Simply: Members, Roles, and Bindings (Without the Jargon) piece is the right warm-up if any of that is shaky, and GCP IAM and Service Accounts: Roles, Bindings and Least Privilege covers the service-account identities that push subscriptions and the Pub/Sub service agent rely on. Knowing HTTP status codes and the idea of an idempotent operation helps, because at-least-once delivery makes idempotency non-negotiable.

This sits at the centre of the serverless and event-driven track. Upstream of it are the compute choices: most modern consumers are Cloud Run Explained: Serverless Containers That Scale to Zero (and Back) Without Kubernetes services receiving push, and the GCP Cloud Run vs GKE vs Compute Engine: Choose the Right Compute decision determines what runs your handlers. Downstream are the sinks: BigQuery for Data Analytics: Warehousing, Querying and Visualization (the destination for a BigQuery subscription) and Cloud Storage Classes Decoded: Standard, Nearline, Coldline, Archive — and Lifecycle Rules (where a Cloud Storage subscription lands batched files). For day-two operations, GCP Cloud Monitoring and Operations: Observability Built In is where you watch subscription backlog and set the alerts this article recommends.

A quick map of who owns which layer of a Pub/Sub system, so an incident reaches the right person fast:

Core concepts

Six mental models make every later decision obvious.

A topic is a named, schema-aware fan-out point; a subscription is one consumer’s durable view of it. A publisher sends a message (a payload of up to 10 MB plus key/value attributes) to a topic. The topic does not store messages for direct reading — instead, every subscription attached to the topic gets its own independent copy of every message published after that subscription was created, with its own backlog, its own retention clock, and its own acknowledgement state. Two subscriptions on one topic are two completely separate streams. This is the whole architecture: publish once, deliver to every subscription, each consumed independently.

Delivery is at-least-once by default, which means duplicates are normal, not exceptional. Pub/Sub guarantees a message is delivered at least once, and will redeliver any message it does not see acknowledged (acked) before the ack deadline expires. Network blips, consumer restarts, a handler that’s slow to ack, even Pub/Sub’s own internal mechanics can cause the same message to arrive two or more times. Therefore every handler must be idempotent — processing the same message twice must produce the same result as processing it once. This single fact drives more correct (and incorrect) Pub/Sub designs than anything else.

The ack deadline is a lease, and you can extend it. When a consumer receives a message, it gets an exclusive lease for the ack deadline (default 10 seconds, configurable 10–600 s). If it acks within the lease, the message is removed from the subscription. If the lease expires first, Pub/Sub assumes the consumer failed and redelivers the message (to that consumer or another). For work that legitimately takes longer than the deadline, the client library issues modifyAckDeadline calls to extend the lease while processing continues — the official client libraries do this automatically up to a maxExtensionPeriod. A deadline shorter than your real processing time is the number-one cause of “why is everything being processed twice?”

Ordering and exactly-once are opt-in and have costs. By default Pub/Sub does not guarantee order and gives at-least-once (so, duplicates). Turn on message ordering and messages sharing an ordering key are delivered in publish order to a given subscriber, at the cost of throughput per key and head-of-line blocking. Turn on exactly-once delivery and Pub/Sub guarantees no acked message is redelivered and gives stronger ack semantics, at the cost of region-pinned subscribers and modestly higher latency. You choose these per subscription, deliberately, when correctness needs them — not by default.

The failure path is first-class: retry policy + dead-letter topic. A message that a consumer keeps nacking (negative-acknowledging) or never acking would otherwise redeliver forever, blocking progress. A retry policy controls the backoff between redeliveries (immediate, or exponential 10 s → 600 s). A dead-letter topic (DLQ) moves a message that has exceeded its maximum delivery attempts (5–100) onto a separate topic, so the poison message is quarantined for inspection and the subscription keeps flowing. Together they turn “one bad message wedges the pipeline” into “bad messages land in a DLQ you alert on.”

Pub/Sub is global; Pub/Sub Lite is zonal and capacity-provisioned. Standard Pub/Sub is a global service: publish in one region, subscribe in another, no capacity planning, pay per data volume. Pub/Sub Lite is a separate, lower-cost product where you provision throughput and storage capacity in a single zone/region — far cheaper at sustained high volume, but with manual capacity management and fewer features. They are different products with the same conceptual model; choosing between them is a real architectural decision covered below.

The vocabulary in one table

Pin down every moving part before the deep sections. The glossary at the end repeats these for lookup; this table is the mental model side by side:

Topics, subscriptions and the publish path

Everything starts with a topic. A topic is cheap, has no fixed throughput to provision, and is global. The first design question is granularity: one topic per event type, or one fat topic with a type attribute? The practitioner default is one topic per logical event (order-placed, order-shipped, payment-captured) because subscriptions, IAM, schemas and DLQs all attach at the topic level — a coarse topic forces every subscriber to filter and shares one access boundary across unrelated events.

Create a topic and publish, both CLI and client code:

# Create a topic with 7-day retention (so subscriptions created later can replay)
gcloud pubsub topics create order-placed \
  --message-retention-duration=7d

# Publish a message with a JSON body and routing attributes
gcloud pubsub topics publish order-placed \
  --message='{"orderId":"A-10231","total":2499,"currency":"INR"}' \
  --attribute=region=in-south,priority=high

# Python: a single, reused PublisherClient (do NOT create one per message)
from google.cloud import pubsub_v1
import json

publisher = pubsub_v1.PublisherClient()
topic_path = publisher.topic_path("my-project", "order-placed")

def publish_order(order: dict) -> str:
    data = json.dumps(order).encode("utf-8")           # body must be bytes
    future = publisher.publish(
        topic_path, data,
        region="in-south", priority="high",            # attributes are kwargs
    )
    return future.result(timeout=30)                   # blocks; returns messageId

Then attach subscriptions. The cardinal rule that surprises everyone: a subscription only receives messages published after it was created. A message published to a topic with zero subscriptions is delivered to nobody and is gone. Create the subscription first, then publish. (For replay of past messages you use retention + seek, covered later — and the subscription must already have existed when those messages were published.)

# A pull subscription with a 30-second ack deadline and 7-day retention
gcloud pubsub subscriptions create inventory-sub \
  --topic=order-placed \
  --ack-deadline=30 \
  --message-retention-duration=7d \
  --expiration-period=never

The topic- and subscription-level knobs you set at creation, with their real defaults and limits:

Hard limits worth memorising, because hitting them is a production incident, not a tuning exercise:

Push vs pull vs streaming pull: the delivery model

How a consumer gets messages is the most consequential subscription choice. There are three subscriber-driven models plus two native sinks; this section covers the three subscriber models and the next covers the native ones.

Pull (unary): the consumer calls Pull to fetch a batch, processes, then calls Acknowledge. Simple, explicit, but a request-per-batch is inefficient at scale. Almost nobody uses raw unary pull directly — the client libraries wrap streaming pull instead.

Streaming pull: the consumer opens a long-lived gRPC stream; Pub/Sub pushes messages down the stream as they arrive, the client library manages leases (auto-extending the ack deadline), flow control, and concurrency. This is what subscriber.subscribe(...) gives you in every official client library. It’s the right default for any consumer you run yourself (GKE pods, Compute Engine, a long-running Cloud Run service with min-instances, a worker process).

Push: Pub/Sub makes an HTTPS POST to your endpoint for each message; you return 2xx to ack, any other status (or a timeout) to nack. No client library, no polling — ideal for serverless handlers (Cloud Run, Cloud Functions) that scale on request volume and to zero. The endpoint must be HTTPS; for security you attach a service account so Pub/Sub sends an OIDC token your endpoint verifies.

The decision, side by side:

Streaming pull in practice

# Streaming pull with the Python client library (handles leases + flow control)
from google.cloud import pubsub_v1
from concurrent.futures import TimeoutError

subscriber = pubsub_v1.SubscriberClient()
sub_path = subscriber.subscription_path("my-project", "inventory-sub")

def callback(message: pubsub_v1.subscriber.message.Message) -> None:
    try:
        order = message.data.decode("utf-8")
        decrement_stock(order)          # your idempotent work
        message.ack()                   # success → remove from backlog
    except TransientError:
        message.nack()                  # fail → redeliver per retry policy
    # an unhandled exception = no ack = redeliver after the ack deadline

# Flow control: never hold more than 100 messages or 50 MB in flight
flow = pubsub_v1.types.FlowControl(max_messages=100, max_bytes=50 * 1024 * 1024)
streaming_future = subscriber.subscribe(sub_path, callback=callback, flow_control=flow)

try:
    streaming_future.result()           # block forever, processing messages
except (KeyboardInterrupt, TimeoutError):
    streaming_future.cancel()
    streaming_future.result()

Push to Cloud Run / Cloud Functions

# A push subscription that POSTs to a Cloud Run service, authenticated with OIDC
gcloud pubsub subscriptions create order-push-sub \
  --topic=order-placed \
  --push-endpoint=https://order-handler-xyz.a.run.app/events \
  --push-auth-service-account=pubsub-push@my-project.iam.gserviceaccount.com \
  --ack-deadline=60

# The Cloud Run handler. 2xx = ack; anything else = nack (redelivered).
import base64, json
from flask import Flask, request

app = Flask(__name__)

@app.post("/events")
def handle():
    envelope = request.get_json(silent=True)
    if not envelope or "message" not in envelope:
        return ("bad request", 400)              # malformed → nack
    msg = envelope["message"]
    data = base64.b64decode(msg["data"]).decode() if msg.get("data") else ""
    order = json.loads(data)
    try:
        process_order(order)                     # idempotent
        return ("", 204)                         # ACK
    except TransientError:
        return ("retry later", 503)              # NACK → Pub/Sub retries

The push payload always wraps your message in an envelope: {"message": {"data": "<base64>", "attributes": {...}, "messageId": "...", "publishTime": "..."}, "subscription": "..."}. Decode data from base64; read routing from attributes. The OIDC token arrives in the Authorization: Bearer header — Cloud Run verifies it automatically when the service requires authentication and the push service account has roles/run.invoker.

Push-specific behaviour and gotchas you must plan for:

BigQuery and Cloud Storage subscriptions: no-code sinks

Two subscription types deliver directly to a Google sink with no consumer code at all — Pub/Sub itself writes the data. They replace the once-universal “Dataflow streaming insert” or “tiny Cloud Function that writes to BigQuery” patterns for the common case, removing an entire moving part.

BigQuery subscription: messages are written straight into a BigQuery table. You either map message attributes/JSON to columns, or land the whole message in a wide schema. No Dataflow job, no compute to run — Pub/Sub streams into BigQuery and you pay (reduced) Pub/Sub data throughput plus BigQuery storage. It’s the fastest path from “events on a topic” to “queryable in BigQuery.”

# Stream a topic straight into a BigQuery table, using the table's schema
gcloud pubsub subscriptions create orders-to-bq \
  --topic=order-placed \
  --bigquery-table=my-project:analytics.orders \
  --use-topic-schema \
  --write-metadata

Cloud Storage subscription: Pub/Sub batches messages and writes them as files (Avro or text) to a GCS bucket, rolling files by size or time. Ideal for cheap archival, a data-lake landing zone, or batch downstreams. You pick the batch trigger (e.g. every 5 minutes or every 100 MB).

# Batch messages into 5-minute / 100 MB files in a GCS bucket as Avro
gcloud pubsub subscriptions create orders-to-gcs \
  --topic=order-placed \
  --cloud-storage-bucket=my-project-events-lake \
  --cloud-storage-file-prefix=orders/ \
  --cloud-storage-max-duration=300s \
  --cloud-storage-max-bytes=100000000 \
  --cloud-storage-output-format=avro

When to reach for each delivery target — the full menu:

For the BigQuery and Cloud Storage subscriptions, Pub/Sub’s own service agent does the writing, so it needs IAM on the destination — a step people forget, producing a healthy-looking subscription whose backlog silently climbs because every write is denied:

The Pub/Sub service agent is service-<PROJECT_NUMBER>@gcp-sa-pubsub.iam.gserviceaccount.com — grant it, not your own account, the destination role.

Message ordering with ordering keys

By default, Pub/Sub does not guarantee order. Under fan-out, retries and parallelism, message 2 can arrive before message 1. For many event systems that is fine — an order-placed and an order-cancelled for different orders have no relative order that matters. But within a single entity (one order, one device, one account) you often need events applied in publish order, or state goes wrong (apply cancelled before placed and you cancel nothing, then create a ghost).

Ordering keys solve this. Tag related messages with the same ordering key (e.g. the order ID), enable ordering on the subscription, and Pub/Sub delivers messages with the same key in the order they were published, to a single subscriber, one at a time — the next is not delivered until the current one is acked. Messages with different keys are still parallel and unordered relative to each other, so throughput scales with the number of distinct keys.

# Ordering must be enabled on the SUBSCRIPTION
gcloud pubsub subscriptions create order-events-ordered \
  --topic=order-events \
  --enable-message-ordering \
  --ack-deadline=30

# Publisher MUST set enable_message_ordering AND publish to the same region endpoint
from google.cloud import pubsub_v1

opts = pubsub_v1.types.PublisherOptions(enable_message_ordering=True)
publisher = pubsub_v1.PublisherClient(
    publisher_options=opts,
    client_options={"api_endpoint": "asia-south1-pubsub.googleapis.com:443"},
)
topic = publisher.topic_path("my-project", "order-events")

def publish_event(order_id: str, body: bytes):
    # All events for one order share the ordering key → delivered in order
    return publisher.publish(topic, body, ordering_key=order_id)

The mechanics and the costs you accept by enabling ordering:

Choosing the ordering key is the whole game. The key should be the identity of the thing whose events must be ordered, at the highest cardinality that still preserves the needed order:

Exactly-once delivery: when at-least-once isn’t enough

At-least-once + idempotent handlers is the default and is correct for most systems. But some operations are genuinely hard to make idempotent (a non-idempotent legacy API, a “send exactly one email,” a financial side-effect with no natural dedup key), and adding a dedup store (Redis/Firestore keyed on messageId) is real work. For those, exactly-once delivery moves the guarantee into Pub/Sub.

Enabling it changes three things: (1) an acknowledged message is never redelivered; (2) ack/nack become reliable operations with status — ack() can succeed or fail, and you only consider work complete on a successful ack; (3) the subscriber is pinned to the message-storage region. Within a subscription, you still might receive a message more than once before it’s acked (a redelivery of an unacked message), but once acked it’s gone for good — so you de-duplicate only across in-flight redeliveries (which the library and messageId handle), not across the whole stream.

gcloud pubsub subscriptions create payments-eo-sub \
  --topic=payment-capture \
  --enable-exactly-once-delivery \
  --ack-deadline=60

# With exactly-once, ack() returns a future you should check
from google.cloud import pubsub_v1

def callback(message):
    capture_payment(message.data)          # non-idempotent side-effect
    ack_future = message.ack_with_response()
    try:
        ack_future.result()                # confirm the ack actually committed
    except pubsub_v1.exceptions.AcknowledgeError as e:
        # ack failed (e.g. lease expired) — the message may be redelivered;
        # do NOT treat the work as durably complete
        log.warning("ack failed: %s", e.error_code)

At-least-once vs exactly-once, decided honestly:

The honest rule: prefer idempotent handlers and at-least-once. Reach for exactly-once when the side-effect is truly non-idempotent and a dedup store is more complex or expensive than accepting the region-pinning and latency cost. It is a powerful feature, not a default to sprinkle everywhere.

Dead-letter topics and retry policy: the failure path

A subscription with no failure plan has exactly one: retry forever. A message your consumer can never process (malformed JSON, a referenced record that no longer exists, a bug that throws on one shape of input) will be nacked or time out, redelivered, fail again, forever — consuming quota, polluting logs, and (with ordering on) blocking its key permanently. Production subscriptions need two settings to make failure survivable.

Retry policy controls the backoff between redeliveries. The default is effectively immediate redelivery, which turns a transient downstream outage into a redelivery storm. Set exponential backoff so retries space out (10 s, then doubling up to 600 s), giving a flapping dependency room to recover:

gcloud pubsub subscriptions update inventory-sub \
  --min-retry-delay=10s \
  --max-retry-delay=600s

Dead-letter topic moves a message that has been delivered max-delivery-attempts times (5–100) onto a separate topic, so it’s quarantined and the subscription keeps flowing. You then attach a subscription to the DLQ to inspect, alert, and reprocess:

# 1) A topic to hold poison messages
gcloud pubsub topics create order-placed-dlq

# 2) Point the main subscription at it, after 5 attempts
gcloud pubsub subscriptions update inventory-sub \
  --dead-letter-topic=order-placed-dlq \
  --max-delivery-attempts=5

# 3) The Pub/Sub service agent must be allowed to publish to the DLQ
#    AND to ack on the source subscription
PROJECT_NUMBER=$(gcloud projects describe my-project --format='value(projectNumber)')
SA="serviceAccount:service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com"

gcloud pubsub topics add-iam-policy-binding order-placed-dlq \
  --member="$SA" --role=roles/pubsub.publisher
gcloud pubsub subscriptions add-iam-policy-binding inventory-sub \
  --member="$SA" --role=roles/pubsub.subscriber

The DLQ machinery, setting by setting:

Crucially, a message in the DLQ carries attributes identifying where it came from, so a reprocessing job (or a human after a fix) can replay it onto the original topic. Treat the DLQ subscription’s backlog as an alert: a non-zero, growing DLQ means messages are being permanently dropped from the main flow and need eyes.

A reprocessing sketch — drain the DLQ back onto the source topic after fixing the bug:

# Pull from the DLQ, re-publish to the original topic, ack the DLQ message
def reprocess_dlq(dlq_sub_path, source_topic_path, subscriber, publisher):
    resp = subscriber.pull(subscription=dlq_sub_path, max_messages=100)
    for rm in resp.received_messages:
        publisher.publish(source_topic_path, rm.message.data,
                          **dict(rm.message.attributes)).result()
        subscriber.acknowledge(subscription=dlq_sub_path,
                               ack_ids=[rm.ack_id])

Schemas: enforcing the message contract

A topic without a schema accepts any bytes, so a producer bug (renamed field, wrong type, truncated JSON) sails through publish and explodes in every consumer — far from the source, multiplied by fan-out. Schemas push the contract to the topic: attach an Avro or Protocol Buffer schema, and Pub/Sub validates every publish against it, rejecting non-conforming messages at the door.

# 1) Define an Avro schema
cat > order.avsc <<'EOF'
{ "type": "record", "name": "Order", "fields": [
  { "name": "orderId",  "type": "string" },
  { "name": "total",    "type": "double" },
  { "name": "currency", "type": "string", "default": "INR" }
]}
EOF
gcloud pubsub schemas create order-schema --type=avro --definition-file=order.avsc

# 2) Bind it to a topic and require JSON-or-binary encoding
gcloud pubsub topics create order-placed-v2 \
  --schema=order-schema \
  --message-encoding=json

Now a publish missing orderId or with a string total is rejected with an error, at the producer, before any consumer is touched. The contract is enforced once, centrally.

Schema choices and how to evolve them without a flag day:

The compatibility rules you must respect so old and new subscribers coexist:

Practical rule: only ever add optional fields with defaults, version with schema revisions when you must break, and never reuse a Protobuf tag or rename an Avro field in place. Schema-on-topic is the single highest-leverage reliability control in a multi-team event system, because it stops bad data at the one place it’s cheap to stop.

Pub/Sub vs Pub/Sub Lite, and Dataflow integration

Pub/Sub Lite is a distinct product: a cheaper, capacity-provisioned messaging service for high, predictable throughput where you accept operational ownership. With Lite you create a reservation/topic and provision publish and subscribe throughput (MiB/s) and storage (GiB) yourself, in a single zone or region. At sustained high volume it can be dramatically cheaper than Pub/Sub — but you plan capacity, you live in one zone/region, and you give up push subscriptions, the BigQuery/GCS native subscriptions, and several other features.

The decision rule: start on Pub/Sub. Move a stream to Lite only when it has steady, high, predictable throughput and the per-GB Pub/Sub bill is material — and you’re willing to own capacity. Most teams never need Lite; a few high-volume telemetry/log streams save a lot with it.

Dataflow is the heavy-transform consumer. When you need windowed aggregations, streaming joins, deduplication at scale, late-data handling, or exactly-once stream processing into BigQuery, a Dataflow (Apache Beam) pipeline reads the Pub/Sub subscription as an unbounded source. Beam’s PubsubIO understands message IDs (for dedup), attributes, and event-time timestamps, and Dataflow gives exactly-once processing semantics within the pipeline.

# Apache Beam (Dataflow): read a subscription, window, write to BigQuery
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions, StandardOptions

opts = PipelineOptions(streaming=True)
opts.view_as(StandardOptions).runner = "DataflowRunner"

with beam.Pipeline(options=opts) as p:
    (p
     | "Read" >> beam.io.ReadFromPubSub(
            subscription="projects/my-project/subscriptions/orders-dataflow")
     | "Parse" >> beam.Map(parse_order)
     | "Window" >> beam.WindowInto(beam.window.FixedWindows(60))   # 1-min windows
     | "Sum" >> beam.CombinePerKey(sum)
     | "ToBQ" >> beam.io.WriteToBigQuery(
            "my-project:analytics.order_totals_1m",
            write_disposition=beam.io.BigQueryDisposition.WRITE_APPEND))

When to choose which consumer for analytics ingestion:

Flow control and subscriber tuning

A subscriber that pulls faster than it can process will OOM, thrash, or exceed ack deadlines and trigger mass redelivery. Flow control is the client-side cap on how much work is outstanding (received but not yet acked) at once — by message count and/or bytes. It is the backpressure valve every self-run consumer needs.

The library holds at most max_messages / max_bytes in flight, stops requesting more until you ack some, and meanwhile auto-extends the ack deadline of held messages (up to max_extension_period). Set these to what one worker can actually process concurrently, not to a guess:

The failure these settings prevent — and the symptom if you get them wrong:

Event-driven patterns on Pub/Sub

The primitives compose into a handful of patterns you’ll reuse across systems. Knowing them by name shortcuts design discussions.

Two of these deserve code-level notes. Routing by attribute uses subscription filters, set at creation and immutable, so each consumer only pays for and processes its slice:

# Only deliver high-priority, India-south messages to this subscription
gcloud pubsub subscriptions create urgent-in-sub \
  --topic=order-placed \
  --message-filter='attributes.priority = "high" AND attributes.region = "in-south"'

Event replay uses snapshots and seek. A snapshot captures a subscription’s ack state at a point in time; seeking the subscription to that snapshot (or to a timestamp within retention) re-delivers everything since:

# Snapshot now, then later rewind the subscription to reprocess from this point
gcloud pubsub snapshots create before-deploy --subscription=inventory-sub
# ... deploy a fix ...
gcloud pubsub subscriptions seek inventory-sub --snapshot=before-deploy
# Or rewind to a wall-clock time (within retention):
gcloud pubsub subscriptions seek inventory-sub --time=2026-06-23T09:00:00Z

For replay to reach back before messages were acked, the subscription needs --retain-acked-messages and sufficient --message-retention-duration. Seek is the feature that lets you reprocess history after fixing a consumer bug or onboarding a brand-new consumer that needs to backfill.

Architecture at a glance

The first diagram shows the fan-out that is the heart of event-driven Pub/Sub. A single publisher (the checkout service) sends one order-placed message to one topic. The topic does no work of its own except branch: every subscription attached to it receives an independent copy. Follow the three branches — an inventory subscription feeding a worker that decrements stock, an email subscription feeding a notification handler, and an analytics subscription feeding the data path. Each subscription holds its own backlog and ack state, so the slow branch (analytics) backs up in its own queue while the fast branches (inventory, email) are untouched. That isolation — one publish, N independent consumers, no shared fate — is the entire value proposition in one picture.

The second diagram traces a concrete end-to-end event flow: a file lands in Cloud Storage, which emits an event that flows through Pub/Sub to a consumer that loads the data into BigQuery. Read it left to right — the GCS upload is the producer event, Pub/Sub is the durable decoupling buffer that absorbs bursts and retries on failure, and BigQuery is the analytical sink. This is the canonical ingestion shape: an operational event (an upload, a transaction, a device reading) becomes a durable message, and a consumer (or a native BigQuery subscription) turns it into queryable data — the operational side stays fast while the analytical side is eventually consistent and independently scalable.

The shared lesson of both diagrams: Pub/Sub sits between a producer and one-or-more consumers as a durable, elastic buffer, and the unit of independence is the subscription. Design topic boundaries around events, subscription boundaries around consumers, and the rest of the system inherits decoupling for free.

Real-world scenario

Kanban Kart, a mid-market Indian e-commerce company, ran the synchronous checkout described in the intro: a Cloud Run checkout service that, inside the customer’s request, called inventory, payments, an email provider, and an analytics ingest endpoint in sequence. Peak traffic was ~600 orders/minute, spiking to ~4,000/minute during festival sales. The platform team was five engineers; monthly spend on the order path was about ₹40,000.

The incident hit on a Diwali sale. At 10:42, the analytics ingest endpoint — a small service writing to BigQuery via streaming inserts — hit a quota limit and began returning 503s with 7-second timeouts. Because checkout awaited analytics last in the chain, every order request now blocked for 7 seconds before failing. Cloud Run scaled checkout to its max instances, but each instance was stuck waiting on analytics, so concurrency saturated and healthy orders started returning 500. Payments and inventory were fine; the least important consumer had taken down the storefront at peak revenue. The on-call’s first reflex — bump Cloud Run max-instances — made it worse (more instances, all stuck on analytics). Twenty-two minutes of partial checkout outage during a sale is real money.

The breakthrough was recognising the coupling, not the capacity. The fix, shipped over the following sprint, was Pub/Sub fan-out. Checkout was changed to do exactly one thing after persisting the order: publish an order-placed message to a topic, then return 201 to the customer. That call is single-digit milliseconds and never waits on a downstream. Four subscriptions were created: inventory (push to a Cloud Run service, ack deadline 30 s), payments (push, exactly-once delivery enabled because a double-capture is unforgiving, ordering key orderId so per-order events stay ordered), email (push to a notification service), and analytics (a BigQuery subscription — no code at all, Pub/Sub streams straight into the warehouse). Every subscription got a dead-letter topic with max-delivery-attempts=10 and exponential retry (10 s → 600 s).

The payoff showed at the next sale. A burst of 4,000 orders/minute published in milliseconds; checkout p99 fell from ~900 ms (synchronous) to ~40 ms. When the analytics path again hit a quota wobble, the BigQuery subscription’s backlog grew to ~9,000 messages and drained within minutes once quota recovered — checkout never noticed. A bug in the email service that threw on one malformed address sent ~30 messages to its DLQ; the team fixed the parser, ran the reprocessing job to replay the DLQ, and lost zero notifications. The order path bill rose modestly (Pub/Sub data volume + the BigQuery subscription throughput) to about ₹44,000, but the architecture absorbed a 6× burst with no capacity ticket and no outage.

The incident and fix as a timeline, because the sequence is the lesson:

Advantages and disadvantages

Pub/Sub trades synchronous simplicity for decoupling and elasticity. Weigh it honestly:

The model is right when consumers must react independently, when load is bursty, when you need fan-out or replay, or when you’re streaming to analytics. It is the wrong tool for synchronous request/response where the caller needs an immediate, definitive answer (use a direct API call or RPC), for strict global total ordering at high throughput (the per-key cap bites), and for tiny, low-volume systems where the operational overhead exceeds the benefit. The disadvantages are all manageable — idempotency, a sane ack deadline, a DLQ, an alert on backlog — but only if you know they exist, which is the point of this article.

Hands-on lab

Build a complete fan-out with a dead-letter path, observe ordering and redelivery, and tear it all down. Free-tier-friendly: a few small messages cost effectively nothing; delete at the end. Run in Cloud Shell.

Step 1 — Set your project and enable the API.

gcloud config set project $(gcloud config get-value project)
gcloud services enable pubsub.googleapis.com
PROJECT_ID=$(gcloud config get-value project)
PROJECT_NUMBER=$(gcloud projects describe "$PROJECT_ID" --format='value(projectNumber)')
echo "Project: $PROJECT_ID ($PROJECT_NUMBER)"

Step 2 — Create the topic and a dead-letter topic.

gcloud pubsub topics create lab-orders --message-retention-duration=1d
gcloud pubsub topics create lab-orders-dlq

Expected: two Created topic lines.

Step 3 — Create two subscriptions on the topic (fan-out), one with a DLQ.

# Consumer A: inventory, 20 s ack deadline, dead-letters after 5 attempts
gcloud pubsub subscriptions create lab-inventory \
  --topic=lab-orders --ack-deadline=20 \
  --dead-letter-topic=lab-orders-dlq --max-delivery-attempts=5 \
  --min-retry-delay=10s --max-retry-delay=600s

# Consumer B: analytics, plain pull
gcloud pubsub subscriptions create lab-analytics \
  --topic=lab-orders --ack-deadline=20

Step 4 — Grant the Pub/Sub service agent the DLQ permissions (forgetting this is the #1 lab failure).

SA="serviceAccount:service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com"
gcloud pubsub topics add-iam-policy-binding lab-orders-dlq \
  --member="$SA" --role=roles/pubsub.publisher
gcloud pubsub subscriptions add-iam-policy-binding lab-inventory \
  --member="$SA" --role=roles/pubsub.subscriber

Step 5 — Publish three messages and watch fan-out.

for id in A-1 A-2 A-3; do
  gcloud pubsub topics publish lab-orders \
    --message="{\"orderId\":\"$id\",\"total\":999}" \
    --attribute=priority=high
done

Expected: three messageIds. The same three messages are now waiting in both subscriptions independently.

Step 6 — Pull and ack from one subscription; the other still has all three.

# Pull from inventory and ACK (auto-ack), then confirm analytics is untouched
gcloud pubsub subscriptions pull lab-inventory --limit=10 --auto-ack
gcloud pubsub subscriptions pull lab-analytics --limit=10   # NO --auto-ack: leaves them

Expected: lab-inventory returns the three messages and removes them; lab-analytics also returns three (proving fan-out gives each subscription its own copy). Because you didn’t --auto-ack the analytics pull, those remain for redelivery.

Step 7 — Observe redelivery and the delivery-attempt counter. Pull lab-analytics again without acking, a few times; note the messages keep coming back (ack deadline expires → redelivery). On lab-inventory, if you repeatedly pull-without-ack, after 5 attempts the message moves to lab-orders-dlq:

# Pull WITHOUT auto-ack repeatedly to force redelivery on inventory
for i in 1 2 3 4 5 6; do
  gcloud pubsub subscriptions pull lab-inventory --limit=10 \
    --format='value(message.attributes.googclient_deliveryattempt, message.data)'
  sleep 22   # let the 20 s ack deadline expire so it redelivers
done
# After ~5 attempts the message should appear in the DLQ:
gcloud pubsub subscriptions create lab-dlq-reader --topic=lab-orders-dlq
gcloud pubsub subscriptions pull lab-dlq-reader --limit=10 --auto-ack

Expected: the delivery-attempt value climbs; once it exceeds max-delivery-attempts=5, the message surfaces in lab-orders-dlq — exactly how poison messages are quarantined in production.

Step 8 — Validation checklist.

Step 9 — Teardown (avoid lingering charges, though they’d be tiny).

gcloud pubsub subscriptions delete lab-inventory lab-analytics lab-dlq-reader --quiet
gcloud pubsub topics delete lab-orders lab-orders-dlq --quiet

Cost note. A handful of small messages and a day of retention cost a fraction of a rupee; deleting the topics and subscriptions stops everything. Pub/Sub has a generous free tier (the first 10 GiB of throughput per month is free), so this lab is effectively free.

Common mistakes & troubleshooting

The failures that actually page you, as a scannable table first, then the worst offenders expanded.

The ones that cause the most wasted hours, expanded:

1. Everything is processed two or three times. The classic. Your handler takes 25 seconds, the ack deadline is the default 10. The lease expires mid-processing, Pub/Sub redelivers, and you do the work again — possibly while the first attempt is still running. Confirm: the subscription’s expired_ack_deadlines_count is non-zero and your handler’s p99 exceeds the ack deadline. Fix: set the ack deadline just above your real p99 (--ack-deadline=60 etc.), let the client library extend the lease for legitimately long work (max_extension_period), and — because at-least-once never fully eliminates duplicates — make the handler idempotent (dedup on messageId, or use a natural idempotency key). Raising the deadline reduces duplicates; idempotency makes the residual ones harmless.

3. The backlog grows forever while the consumer “looks fine.” A consumer that swallows an exception and never acks, or a push endpoint quietly returning 500, keeps every message unacked, so num_undelivered_messages climbs without bound. Confirm: oldest_unacked_message_age and num_undelivered_messages rising on the subscription (Cloud Monitoring); for push, the endpoint’s request logs show non-2xx. Fix: ensure the handler acks only on success and nacks/throws on failure; for push, fix whatever makes the endpoint return non-2xx. A growing oldest_unacked_message_age is the single most important Pub/Sub alarm — wire it.

4. A BigQuery or Cloud Storage subscription backs up and there are no errors in your code — because there is no code. Native subscriptions are written by the Pub/Sub service agent, so a missing IAM grant on the destination silently fails every write while the subscription looks healthy. Confirm: the subscription’s backlog climbs; the sink shows no new rows/files; the service agent lacks the role. Fix: grant service-<PROJECT_NUMBER>@gcp-sa-pubsub.iam.gserviceaccount.com the destination role (roles/bigquery.dataEditor on the dataset, roles/storage.objectCreator on the bucket). This is the most surprising failure precisely because there’s no application log to look at.

9. A single poison message wedges the whole subscription. Without a DLQ, a message your consumer can never process is nacked, redelivered, fails, forever — and with ordering on, it blocks its entire key. Confirm: the same messageId reappears indefinitely; with ordering, a key stops progressing. Fix: attach a dead-letter topic with a sane max-delivery-attempts (5–20), grant the service agent the DLQ permissions, and alert on DLQ backlog. The poison message lands in the DLQ after N tries and the main flow resumes.

Best practices

• Make every handler idempotent. At-least-once means duplicates are normal. Dedup on messageId or a natural key, or use exactly-once where dedup is impractical. This is the non-negotiable foundation.
• Set the ack deadline just above your real p99 processing time, and let the client library extend leases for legitimately long work. A too-short deadline is the top cause of duplicate processing.
• Give every production subscription a dead-letter topic with a tuned max-delivery-attempts, and alert on DLQ backlog. A poison message must never be able to wedge the flow.
• Set an exponential retry policy (min-retry-delay/max-retry-delay) so a transient downstream outage doesn’t become a redelivery storm.
• One topic per logical event type. Subscriptions, IAM, schemas and DLQs attach at the topic; coarse topics force filtering and share access boundaries across unrelated events.
• Create subscriptions before you publish. A message published to a topic with no matching subscription is gone. For replay, enable retention (and retain-acked-messages if you need to rewind past acks).
• Use ordering keys only when order truly matters, and choose a high-cardinality key (entity ID, not region/type) so you don’t serialise the whole stream onto one ~1 MB/s key.
• Attach a schema to every shared topic and only ever add optional fields with defaults; version with revisions when you must break. Stop bad data at the source.
• Prefer native subscriptions (BigQuery, Cloud Storage) over hand-rolled glue for plain sink loads — fewer moving parts, and Pub/Sub does the writing.
• Tune flow control to your handler’s real concurrency so a fast pull doesn’t OOM a slow consumer and trigger mass redelivery.
• Grant the Pub/Sub service agent the right destination IAM for push (OIDC + run.invoker), BigQuery and GCS subscriptions, and DLQs — the most common “healthy but silently broken” failure.
• Alert on oldest_unacked_message_age and num_undelivered_messages, not just “is the consumer up.” Backlog age is the truest health signal.

The alerts worth wiring before the next incident — leading indicators, not “the service is down”:

Security notes

• Least-privilege roles, not broad ones. Grant roles/pubsub.publisher to producers and roles/pubsub.subscriber to consumers on the specific topic/subscription — never roles/pubsub.admin or roles/editor to an app. The granular roles exist precisely so a compromised consumer can’t create topics or read others.
• Use a dedicated service account per workload. The publisher, each consumer, and the push identity should be distinct service accounts so you can revoke and audit independently. Workload Identity (GKE) or attached service accounts (Cloud Run/GCE) avoid key files entirely.
• Secure push endpoints with OIDC. Set --push-auth-service-account, require authentication on the Cloud Run service, and grant that SA roles/run.invoker. Without this, your push endpoint is an open, unauthenticated webhook anyone can POST to.
• Don’t put secrets in messages or attributes. Messages may be retained for days and copied to every subscription, DLQ, and (for native subs) a sink. Put references, not credentials, in payloads.
• Constrain the service agent. The Pub/Sub service agent needs only the specific destination roles (DLQ publish, BigQuery dataEditor, GCS objectCreator) — grant exactly those, on exactly those resources, not project-wide.
• CMEK for sensitive data. Pub/Sub supports customer-managed encryption keys; use them where compliance requires control over the encryption key for messages at rest.
• VPC Service Controls for exfiltration risk. For regulated data, place Pub/Sub inside a service perimeter so messages can’t be published or pulled across the perimeter boundary.

The security controls and what each prevents:

Cost & sizing

Pub/Sub billing is driven by data volume, not by message count or topic/subscription count, plus storage for retention and any seek/snapshot retention. The mental model: you pay for throughput (bytes published, and bytes delivered — fan-out multiplies delivery), and for retained storage (longer retention = more storage). There is a generous free tier (the first 10 GiB of throughput per month), which makes small systems effectively free.

• Throughput dominates at scale. Every published byte is billed once on publish; every delivery is billed, so a topic with five subscriptions delivers (and bills) roughly five times the published volume. High fan-out is powerful but multiplies the delivery bill — consolidate where consumers don’t truly need independence.
• Retention is storage. Default 7-day retention is cheap for modest volume; 31-day retention on a high-volume topic is real money. Set retention to what you actually need for replay/audit, not the max “just in case.”
• Native subscriptions have their own economics. BigQuery and Cloud Storage subscriptions bill at a reduced Pub/Sub throughput rate plus the destination’s storage — usually cheaper than running a Dataflow job or a custom consumer for the same load.
• Dataflow is a separate, often larger bill. A streaming Dataflow pipeline runs workers continuously; reserve it for transforms native subscriptions can’t do.
• Pub/Sub Lite is the cost lever at sustained high volume. If a stream is steady and high-throughput, provisioned Lite capacity can be several times cheaper than per-GB Pub/Sub — at the cost of capacity management and lost features.

A rough monthly picture for a mid-size event system (~50 GB/month published, 3 subscriptions, 7-day retention): well within or just past the free tier on throughput, with delivery ≈ 3× published — order of a few hundred to low thousands of rupees for Pub/Sub itself, dwarfed by the compute consuming it. The cost drivers and what each buys:

Interview & exam questions

1. What is the difference between a topic and a subscription, and how does fan-out work? A topic is the named fan-out point publishers send to; a subscription is one consumer’s durable, independent view of that topic. Every subscription gets its own copy of every message published after the subscription was created, with its own backlog and ack state. Fan-out is simply attaching multiple subscriptions to one topic — publish once, each subscription consumes independently, so a slow consumer can’t affect the others.

2. Pub/Sub delivers at-least-once by default. What does that force on your design? That duplicates are normal — network blips, consumer restarts, or a slow ack cause the same message to be redelivered. Every handler must therefore be idempotent: processing a message twice yields the same result as once. You either dedup on messageId/a natural key, or enable exactly-once delivery (accepting its region-pinning and slightly higher latency).

3. A consumer’s work takes 30 seconds but messages are processed two or three times. Why, and the fix? The ack deadline (default 10 s) is shorter than the processing time, so the lease expires mid-processing and Pub/Sub redelivers. Fix by raising the ack deadline above your real p99 and letting the client library extend the lease (max_extension_period) for long work; keep handlers idempotent because at-least-once never fully eliminates duplicates.

4. How do ordering keys work and what do they cost? Messages sharing an ordering key are delivered in publish order to a single subscriber, one at a time, when ordering is enabled on both the subscription and publisher (and the publisher pins to a region endpoint). The cost is throughput — roughly 1 MB/s per key — plus head-of-line blocking, so a stuck message stalls its key. Choose a high-cardinality key (entity ID) so distinct keys still parallelise.

5. When would you enable exactly-once delivery instead of relying on idempotency? When the side-effect is genuinely hard to make idempotent (a non-idempotent legacy API, a single email, an unforgiving financial action) and a dedup store is more complex/expensive than the constraints exactly-once imposes (subscriber pinned to the message-storage region, slightly higher latency). Exactly-once guarantees an acked message is never redelivered and gives reliable ack status.

6. What is a dead-letter topic and why is it essential in production? A DLQ is a separate topic a message is moved to after exceeding max-delivery-attempts (5–100), quarantining a poison message so it stops being redelivered forever and the subscription keeps flowing. Without it, an unprocessable message retries indefinitely (and, with ordering, blocks its key). You attach a subscription to the DLQ to inspect, alert, and reprocess.

7. Compare push and pull subscriptions. With pull/streaming pull, the consumer fetches messages (the client library opens a stream, manages leases and flow control) — best for self-run, high-throughput workers. With push, Pub/Sub POSTs each message to your HTTPS endpoint and a 2xx acks it — best for serverless handlers (Cloud Run/Functions) that scale to zero. Push throughput is bounded by your endpoint; pull scales with parallel streams.

8. What are BigQuery and Cloud Storage subscriptions, and what do they replace? They deliver messages directly to a sink with no consumer code — Pub/Sub writes rows to a BigQuery table or batched files to a GCS bucket. They replace the old pattern of a Dataflow job or a small Cloud Function whose only job was to copy events into the warehouse or lake. They need the Pub/Sub service agent to hold IAM on the destination.

9. How do schemas help, and what schema changes are safe? A schema (Avro/Protobuf) bound to a topic makes Pub/Sub validate every publish, rejecting bad payloads at the producer before fan-out multiplies the damage. Safe changes: adding optional fields with defaults; unsafe: adding required fields, renaming, or changing types. Version breaking changes with schema revisions.

10. Pub/Sub vs Pub/Sub Lite — when each? Pub/Sub is global, auto-scaled, feature-rich (push, native sinks, exactly-once), billed per data volume — the default for most workloads. Pub/Sub Lite is zonal/regional, capacity-provisioned (you size throughput and storage), much cheaper at sustained high volume, but with manual capacity management and fewer features. Use Lite only for steady, very high-throughput streams where the per-GB bill is material.

11. Why might a BigQuery subscription show a growing backlog with no errors in your code? Because there is no code — Pub/Sub’s service agent does the writing, so a missing IAM grant on the destination silently fails every write while the subscription looks healthy. Grant service-<PROJECT_NUMBER>@gcp-sa-pubsub.iam.gserviceaccount.com roles/bigquery.dataEditor on the dataset. The same applies to GCS subscriptions (storage.objectCreator) and DLQs (pubsub.publisher).

12. How do you reprocess events after fixing a consumer bug? Use snapshots and seek: snapshot the subscription’s ack state (or rely on retention), deploy the fix, then seek the subscription back to the snapshot or a timestamp within the retention window to redeliver everything since. For replay to reach before acks, enable retain-acked-messages and sufficient retention. This is how you backfill a new consumer or recover from a processing bug without data loss.

These map to the Google Cloud Professional Cloud Architect and Professional Data Engineer exams (event-driven design, streaming ingestion, Dataflow, exactly-once, BigQuery streaming), and to the Associate Cloud Engineer for the core Pub/Sub mechanics and IAM. A compact cert mapping:

Quick check

1. You attach a third subscription to an existing topic and it receives none of the messages published last week. Why, and what mechanism would let you replay them?
2. A consumer’s handler takes 40 seconds, and you’re seeing every message processed twice. What single setting is wrong, and what else must the handler be?
3. A Cloud Storage subscription’s backlog is climbing but your application logs show nothing — because there are none. What’s the most likely cause?
4. You enable message ordering on the subscription but messages still arrive out of order. What did you forget?
5. True or false: a dead-letter topic is optional in production because Pub/Sub retries failed messages anyway.

Answers

1. A subscription only receives messages published after it was created, so last week’s messages were never copied to it. To replay them, the messages must still be within the topic/subscription retention window and a subscription that existed at publish time can seek to a past timestamp; a brand-new subscription cannot retrieve messages published before it existed. (Plan retention and, if needed, retain-acked-messages ahead of time.)
2. The ack deadline is too short (default 10 s < 40 s processing), so the lease expires mid-processing and Pub/Sub redelivers. Raise the ack deadline above your real p99 and rely on the client library’s lease extension — and because at-least-once never fully eliminates duplicates, the handler must also be idempotent.
3. The Pub/Sub service agent (service-<PROJECT_NUMBER>@gcp-sa-pubsub.iam.gserviceaccount.com) lacks IAM on the bucket. Native subscriptions are written by the service agent, not your code, so a missing roles/storage.objectCreator fails every write silently while the subscription looks healthy. Grant it the role on the bucket.
4. The publisher side — ordering must be enabled on the publisher (enable_message_ordering) and the publisher must pin to a single region endpoint, and the messages must share an ordering key. Enabling it only on the subscription is not enough.
5. False. Without a DLQ, a message the consumer can never process retries forever, consuming quota, polluting logs, and (with ordering) blocking its key. A dead-letter topic with max-delivery-attempts quarantines it after N tries so the main flow keeps moving.

Glossary

• Topic — the named, global fan-out point publishers send messages to; subscriptions branch from it.
• Subscription — one consumer’s durable, independent view of a topic, with its own backlog, retention and ack state.
• Message — the unit of delivery: a payload (≤10 MB) plus up to 100 key/value attributes and an optional ordering key.
• Attributes — string key/value metadata on a message, usable for routing, filtering and context without parsing the body.
• Publisher / Subscriber — the code that sends to a topic / receives from a subscription; fully decoupled from each other.
• Ack / Nack — acknowledge (processed, remove) / negative-acknowledge (failed, redeliver now) a received message.
• Ack deadline — the lease window (10–600 s, default 10) to ack before Pub/Sub redelivers; the client library can extend it.
• At-least-once delivery — the default guarantee that a message is delivered one or more times; duplicates are possible, so handlers must be idempotent.
• Exactly-once delivery — an opt-in per-subscription guarantee that an acked message is never redelivered, with reliable ack status; pins the subscriber to the message-storage region.
• Ordering key — a tag that forces messages sharing it to be delivered in publish order to a single subscriber, at ~1 MB/s per key.
• Streaming pull — a long-lived gRPC stream down which Pub/Sub pushes messages to a client library that manages leases and flow control.
• Push subscription — Pub/Sub POSTs each message to your HTTPS endpoint; a 2xx response acks it.
• BigQuery / Cloud Storage subscription — native subscriptions where Pub/Sub writes directly to a BQ table or GCS files with no consumer code.
• Dead-letter topic (DLQ) — a separate topic a message is moved to after max-delivery-attempts, quarantining poison messages.
• Retry policy — the backoff (immediate or exponential 10–600 s) between redeliveries of an unacked/nacked message.
• Schema — an Avro or Protobuf contract bound to a topic; Pub/Sub validates every publish against it.
• Flow control — the client-side cap on outstanding (received-but-unacked) messages/bytes; backpressure for self-run consumers.
• Snapshot / Seek — a saved ack state / the operation to rewind a subscription to a snapshot or timestamp for replay.
• Service agent — the Google-managed identity (service-<PROJECT_NUMBER>@gcp-sa-pubsub.iam.gserviceaccount.com) that performs DLQ publishes and native-subscription writes; it needs destination IAM.
• Pub/Sub Lite — a separate, capacity-provisioned, zonal/regional messaging product that is cheaper at sustained high volume but feature-limited.
• Dataflow — the Apache Beam-based streaming engine that reads a subscription for windowed/joined/deduplicated transforms with exactly-once processing.

Next steps

You can now design, secure and operate an event-driven system on Pub/Sub. Build outward:

• Next: Cloud Run Explained: Serverless Containers That Scale to Zero (and Back) Without Kubernetes — the most common push-subscription consumer; scale your event handlers to zero.
• Related: BigQuery for Data Analytics: Warehousing, Querying and Visualization — the destination for a BigQuery subscription and the analytics half of every ingestion flow.
• Related: Cloud Storage Classes Decoded: Standard, Nearline, Coldline, Archive — and Lifecycle Rules — where a Cloud Storage subscription lands batched events, and how lifecycle rules age them cheaply.
• Related: GCP IAM and Service Accounts: Roles, Bindings and Least Privilege — the service accounts and service-agent grants that push, BigQuery and GCS subscriptions depend on.
• Related: GCP Cloud Monitoring and Operations: Observability Built In — where you watch subscription backlog and oldest-unacked-age and wire the alerts this article recommends.
• Related: GCP Cloud Run vs GKE vs Compute Engine: Choose the Right Compute — decide what runs your consumers, from scale-to-zero push handlers to always-on streaming-pull workers.