GCP Enterprise Architecture: Global Web Application

A global web application looks deceptively simple from the browser: a URL, a fast page, a checkout button that works. Behind that calm surface is one of the harder distributed-systems problems in cloud architecture — serving users on five continents from a single logical application, keeping the data consistent enough to take money, and doing it without a global outage every time a region wobbles. This article is a complete, reusable GCP reference architecture for exactly that, built on the Global External Application Load Balancer, Cloud Run (with a GKE variant), Cloud Spanner, and Cloud CDN. It is written to scale down to a Series-B startup and up to a publicly listed enterprise without changing shape.

In a nutshell

Imagine one coffee brand with shops on every continent. A customer never types a different address — they just walk in, and they always end up in the nearest shop. Every shop pulls pre-made pastries straight off the counter (instant, no wait), makes custom drinks fresh to order, and — this is the clever part — writes every sale into one shared ledger that all shops read and write, and that never loses a single entry even if an entire city’s shops shut for the day. A guard at the door checks everyone before they reach the counter.

That is this architecture, and each piece maps to one idea:

• Anycast IP + global load balancer = the one address that always routes you to the nearest shop.
• Cloud CDN = the pastries already on the counter — static files (images, JS/CSS) served instantly from the edge, never touching a server.
• Cloud Run / GKE = the baristas — stateless compute that makes each dynamic response fresh, in the region closest to you, and costs almost nothing when idle.
• Cloud Spanner = the one shared ledger — a single globally consistent database that survives losing a whole region with zero data loss, so there is no sharding and no “which copy is right?”
• Cloud Armor = the guard at the door — a web-application firewall and DDoS shield that filters requests before they reach compute.

Why a beginner should care: this is the shape of nearly every modern global consumer app — a shopping site, a SaaS dashboard, a streaming service. Learn it once and you understand the three tensions every such system must resolve: latency (be near the user), consistency (never take an order twice or lose one), and cost (don’t pay for a global footprint you aren’t using). Everything below is those three tensions, made concrete.

Level: Advanced · Time: ~40 min

Before you start, it helps to be comfortable with a handful of ideas from earlier lessons: what a region and a zone are, what a load balancer and DNS do, what “stateless” means for a service, and the basics of a relational database. If any of the front-door pieces feel fuzzy, the Global External Application Load Balancer deep dive, the Cloud Run deep dive, and the Spanner schema-design lesson each go one level below what we assume here.

After this lesson you will be able to:

• Explain, end to end, how a single request travels from a browser in Frankfurt to a globally consistent write and back.
• Choose correctly between multi-region Spanner, regional Spanner, and Cloud SQL for a given consistency and RPO requirement — and justify the cost.
• Wire the front door in Terraform: regional serverless NEGs behind one global backend service, with Cloud CDN and Cloud Armor.
• Design a Spanner schema (primary keys + interleaving) that scales writes instead of hotspotting.
• Reason about failover, stale-vs-strong reads, and expand-then-contract migrations the way a production team does.

The business scenario

Picture a company that sells something online to a worldwide audience — a SaaS dashboard, a media subscription, a retail storefront, a booking platform. The specifics vary; the pressures do not. Three forces consistently push teams toward this exact architecture.

The first is latency tied directly to revenue. Every study an e-commerce or media team has ever run says the same thing: page latency and conversion move together. A customer in Singapore or São Paulo who waits 1.8 seconds for first contentful paint converts measurably worse than one served in 400 ms. When your application servers and database live in us-central1 and a third of your revenue comes from outside North America, you are quietly taxing your best growth markets with the speed of light. The round trip from Sydney to Iowa is roughly 160 ms one way before your code does anything.

The second is the operational cost of regional sharding. The traditional answer — stand up a full stack per region and shard customers geographically — works until the day a European customer travels to the US, or a B2B account has offices on three continents, or compliance asks “show me this user’s data” and the answer is “which shard were they in last March?” Cross-shard transactions, dual writes, and bespoke reconciliation jobs become a permanent tax on every feature the team ships. Most engineering orgs underestimate this cost by an order of magnitude.

The third is the demand for a single global brand experience that never fully goes down. Modern customers do not accept “the site is down for maintenance in your region.” A payments page that 500s during a regional GCP incident is an incident on the company’s revenue and its reputation. Boards now ask about RTO and RPO for the customer-facing tier the way they used to ask about it only for the back-office ERP.

This architecture solves all three at once. A single anycast IP fronts the application worldwide, so the closest Google edge terminates the connection. Stateless services run in multiple regions and scale to zero when idle, so a small company is not paying for a global footprint it is not using. And Cloud Spanner provides one logical, strongly consistent, horizontally scalable database spanning regions — eliminating sharding logic entirely while surviving the loss of an entire region with zero data loss (RPO = 0). The same blueprint serves a 5,000-user beta and a 50-million-user platform; only the instance sizing and region count change.

Architecture overview

Follow a single request from a user in Frankfurt and the design explains itself.

The browser resolves the application’s hostname to a single global anycast IP address — one address advertised from every Google point of presence on Earth. Because it is anycast, the network routes the user to the nearest Google edge (Frankfurt, in this case), not to wherever the servers happen to live. TLS is terminated at that edge by the Global External Application Load Balancer, a single global resource (not one per region) with a Google-managed certificate covering the apex and wildcard hostnames.

At the edge, the request meets two gatekeepers before it ever touches compute. Cloud Armor evaluates the request against WAF rules (the OWASP-tuned preconfigured rule sets), per-IP and per-token rate limits, and optional geo or bot-management policies. Immediately after, the load balancer consults its URL map. Static and cacheable paths — /_next/static/*, images, JS/CSS bundles, public marketing pages — are served by Cloud CDN directly from the edge cache, so the vast majority of byte volume never travels to a region at all. A cache hit in Frankfurt is answered in Frankfurt.

Dynamic paths — /api/*, the authenticated app shell, the checkout flow — are routed by the URL map to a backend service. That backend service is the load balancer’s most important decision: it is fronted by a serverless network endpoint group (NEG) per region, pointing at Cloud Run services deployed in, say, europe-west1, us-central1, and asia-southeast1. The global load balancer steers each request to the closest healthy region with capacity, automatically failing over to the next-nearest region if one is unhealthy or saturated. The Frankfurt user lands on Cloud Run in europe-west1 — same continent, single-digit-millisecond regional hops.

The Cloud Run service runs the application’s stateless business logic. It reads configuration and secrets from Secret Manager, calls downstream services over the VPC via Serverless VPC Access or Direct VPC egress, and — for the request that matters — reads and writes the Cloud Spanner database. Spanner is the architectural keystone: a multi-region instance (for example, eur6 or nam-eur-asia1) presents itself as one database with one schema and one connection endpoint, while physically replicating synchronously across regions using Paxos. A write to a customer’s order in europe-west1 is committed by a quorum of replicas across regions before the API returns success. The Frankfurt user’s order is durable across a continental failure the instant they see the confirmation.

So the end-to-end flow is: anycast DNS → nearest Google edge → TLS termination → Cloud Armor → URL map → (Cloud CDN cache hit) OR (regional Cloud Run via serverless NEG) → Spanner multi-region. Asynchronous work — emails, search indexing, analytics, fan-out — is published to Pub/Sub and processed by separate Cloud Run jobs or workers, keeping the synchronous request path short. Telemetry from every hop flows into Cloud Logging, Cloud Monitoring, and Cloud Trace.

The diagram, described in words: at the top, many globe-distributed users; below them a single anycast VIP feeding one global front end (Cloud Armor + Cloud CDN + URL map). From the URL map, one arrow goes left to the CDN/edge cache for static content, and one goes down to a backend service that fans out to three regional Cloud Run boxes. All three regional boxes point downward to a single Spanner cylinder drawn straddling all three regions to signify one logical database. Off to the side, Pub/Sub and worker jobs hang off the Cloud Run tier, and a monitoring plane underlays everything.

Component breakdown

Two component choices deserve emphasis because they are where teams most often go wrong.

The backend service is global, singular, and the seam of the whole design. It is tempting to create a load balancer per region and stitch them with DNS. Do not. A single global backend service with regional serverless NEGs is what gives you anycast entry, automatic capacity-aware steering, and instant cross-region failover with no DNS TTL to wait out. The intelligence lives in one object.

Spanner’s schema is a performance decision, not just a data-modeling one. Spanner scales by splitting data across servers on primary-key ranges. A monotonically increasing key (a timestamp or auto-increment ID) funnels all new writes to one split — the classic hotspot that makes a benchmark look terrible and gets blamed on “Spanner being slow.” Use UUIDv4, a hashed prefix, or bit-reversed sequences for high-write tables, and interleave child rows (order line items under an order) so related data is co-located and joins stay local. Get this right on day one; it is painful to change once you have data.

Implementation guidance

Infrastructure as code. Provision everything with Terraform using the Google provider; nothing here should be click-ops. The dependency order that keeps terraform apply clean is:

1. Foundation — project, VPC, subnets per region, Cloud NAT, firewall rules, and the org/IAM scaffolding. Many teams use the Cloud Foundation Toolkit (terraform-google-modules) for this layer so it matches Google’s security baseline.
2. Data — the Spanner instance and database. Apply schema with the native Terraform google_spanner_database ddl list, or keep DDL under a migration tool (Liquibase has a Spanner extension; wrench is the lightweight Spanner-native option). Treat schema changes as versioned migrations in CI.
3. Compute — Cloud Run services per region (google_cloud_run_v2_service), each with its runtime service account, Direct VPC egress, and min/max instances. For the GKE variant, regional Autopilot clusters and their NEG-backed services.
4. Edge — the serverless NEGs (google_compute_region_network_endpoint_group), the global backend service binding all regional NEGs, the URL map, the CDN-enabled static backend bucket/service, the Cloud Armor security policy, the managed certificate, the target HTTPS proxy, and the global forwarding rule.
5. DNS & secrets — the Cloud DNS records pointing at the global IP, and Secret Manager entries (values injected out-of-band, never committed).

Keep state remote in a GCS backend with object versioning and state locking, and split the layers into separate state files (or Terragrunt stacks) so an edge change cannot accidentally destroy the Spanner instance. The reverse of the create order is your safe destroy order. (If your org standardizes elsewhere, the same topology is expressible in Pulumi or Config Connector; Deployment Manager is legacy and not recommended for new builds.)

Networking wiring. Cloud Run reaches private resources through Direct VPC egress (preferred over the older Serverless VPC Access connector — lower latency, no connector instances to size). Egress to the internet for third-party APIs goes through Cloud NAT so you present stable, allowlistable IPs. Spanner is reached over Google’s private network via its API endpoint; lock it down with VPC Service Controls so the database cannot be exfiltrated to a project outside your perimeter even if a credential leaks. Keep the entire backend free of public ingress — the only public surface is the global load balancer’s VIP, and Cloud Run services are set to allow ingress from “internal and Cloud Load Balancing” only.

Identity wiring. Every Cloud Run service and GKE workload runs as a dedicated, least-privilege service account — one per service, never the default compute SA. Grant roles/spanner.databaseUser (not databaseAdmin) on the specific database, and roles/secretmanager.secretAccessor on the specific secrets. For the GKE variant, bind Kubernetes service accounts to Google service accounts with Workload Identity Federation so no JSON keys ever exist. End-user authentication is handled in the app tier — typically Identity Platform (the productized Firebase Auth) or your existing OIDC IdP — and the front door can additionally enforce Identity-Aware Proxy (IAP) on internal/admin paths for a Zero-Trust, identity-aware perimeter. Human and pipeline access to deploy uses Workload Identity Federation from your CI (GitHub Actions/GitLab) — again, no long-lived keys.

Deploy and release. Build images in Cloud Build or your CI, push to Artifact Registry, and roll out with Cloud Deploy across the regions. Cloud Run’s revision-based traffic splitting gives you canary and blue-green for free: send 5% to the new revision, watch SLOs, then ramp. Because the database is a single Spanner instance shared by all regions and revisions, schema migrations must be backward-compatible (expand-then-contract): add columns/tables, deploy code that tolerates both shapes, backfill, then remove the old shape in a later release. Never ship a breaking DDL in lockstep with code across regions.

Seeing it in code — a worked build

Prose tells you what to build; this section shows the load-bearing pieces so you can read the actual shapes. Everything below is illustrative and trimmed for clarity — placeholders like PROJECT_ID and image tags are yours to fill in — but the resource names, fields, and flags are real and current.

1. The seam: regional serverless NEGs behind ONE global backend service. This is the single most important object in the design. Each region gets a serverless NEG pointing at that region’s Cloud Run service; one global backend service binds all of them.

locals { regions = ["europe-west1", "us-central1", "asia-southeast1"] }

# One serverless NEG per region -> that region's Cloud Run service
resource "google_compute_region_network_endpoint_group" "run_neg" {
  for_each              = toset(local.regions)
  name                  = "run-neg-${each.value}"
  region                = each.value
  network_endpoint_type = "SERVERLESS"
  cloud_run {
    service = google_cloud_run_v2_service.app[each.value].name
  }
}

# THE seam: a single GLOBAL backend service binding every regional NEG.
# Cloud Armor attaches here; Cloud CDN is enabled on the separate bucket backend.
resource "google_compute_backend_service" "app" {
  name                  = "app-backend"
  load_balancing_scheme = "EXTERNAL_MANAGED"      # global external ALB (Envoy)
  security_policy       = google_compute_security_policy.armor.id
  # NOTE: serverless-NEG backends take NO health_check and NO balancing_mode.
  # Google health-checks Cloud Run internally and routes to the closest healthy region.
  dynamic "backend" {
    for_each = google_compute_region_network_endpoint_group.run_neg
    content { group = backend.value.id }
  }
}

The comment on the last block matters and is a common source of confusion: serverless-NEG backends do not accept a health check or a balancing mode. You will see balancing_mode, max_rate_per_endpoint, and outlier_detection on instance-group and zonal-NEG backends (the GKE variant), but not here. The “Going deeper” section unpacks exactly how steering differs between the two.

2. Cloud Run per region, LB-only ingress, Direct VPC egress. No public URL; the only way in is the load balancer.

resource "google_cloud_run_v2_service" "app" {
  for_each = toset(local.regions)
  name     = "app"
  location = each.value
  ingress  = "INGRESS_TRAFFIC_INTERNAL_LOAD_BALANCER"   # LB + internal only

  template {
    service_account = google_service_account.run_sa[each.value].email
    scaling {
      # Kill cold starts only where traffic lives; let the rest scale from zero.
      min_instance_count = contains(["europe-west1", "asia-southeast1"], each.value) ? 2 : 0
      max_instance_count = 100
    }
    max_instance_request_concurrency = 80
    containers {
      image = "europe-west1-docker.pkg.dev/PROJECT_ID/app/web:1.0.0"
    }
    vpc_access {                                   # Direct VPC egress (no connector)
      network_interfaces {
        network    = "projects/PROJECT_ID/global/networks/app-vpc"
        subnetwork = "projects/PROJECT_ID/regions/${each.value}/subnetworks/run-${each.value}"
      }
      egress = "PRIVATE_RANGES_ONLY"
    }
  }
}

3. Cloud Armor: WAF rule + per-IP rate-ban. A preconfigured SQLi rule and a rate-based ban, plus the mandatory catch-all default rule.

resource "google_compute_security_policy" "armor" {
  name = "edge-armor"

  rule {                                            # OWASP SQLi (preconfigured WAF)
    action   = "deny(403)"
    priority = 1000
    match { expr { expression = "evaluatePreconfiguredExpr('sqli-v33-stable')" } }
  }

  rule {                                            # 600 req/min per IP, then 10-min ban
    action   = "rate_based_ban"
    priority = 2000
    match {
      versioned_expr = "SRC_IPS_V1"
      config { src_ip_ranges = ["*"] }
    }
    rate_limit_options {
      enforce_on_key   = "IP"
      conform_action   = "allow"
      exceed_action    = "deny(429)"
      ban_duration_sec = 600
      rate_limit_threshold { count = 600  interval_sec = 60 }
    }
  }

  rule {                                            # required default (lowest priority)
    action   = "allow"
    priority = 2147483647
    match {
      versioned_expr = "SRC_IPS_V1"
      config { src_ip_ranges = ["*"] }
    }
  }
}

4. The static path: a CDN-enabled bucket backend and the URL map that splits traffic. Static assets go to the edge cache; everything else falls through to the app backend.

resource "google_compute_backend_bucket" "static" {
  name        = "static-backend"
  bucket_name = google_storage_bucket.static.name
  enable_cdn  = true
  cdn_policy {
    cache_mode        = "CACHE_ALL_STATIC"
    default_ttl       = 3600
    negative_caching  = true
  }
}

resource "google_compute_url_map" "app" {
  name            = "app-urlmap"
  default_service = google_compute_backend_service.app.id     # dynamic -> Cloud Run
  host_rule { hosts = ["app.example.com"]  path_matcher = "main" }
  path_matcher {
    name            = "main"
    default_service = google_compute_backend_service.app.id
    path_rule {                                                # static -> CDN bucket
      paths   = ["/_next/static/*", "/static/*", "/assets/*"]
      service = google_compute_backend_bucket.static.id
    }
  }
}

The managed certificate, target HTTPS proxy, and global forwarding rule complete the front door (google_compute_managed_ssl_certificate → google_compute_target_https_proxy → google_compute_global_forwarding_rule on port 443, EXTERNAL_MANAGED, pointed at a reserved google_compute_global_address). They are mechanical once the pieces above exist.

5. Spanner schema: interleaving + hotspot-safe keys. This is the schema decision that determines whether Spanner flies or crawls. Note the shared key prefix and the INTERLEAVE IN PARENT clauses that co-locate a customer’s orders and line items on the same split.

CREATE TABLE Customers (
  CustomerId  STRING(36) NOT NULL,        -- UUIDv4: spreads writes across splits
  Email       STRING(320) NOT NULL,
  CreatedAt   TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp = true),
) PRIMARY KEY (CustomerId);

CREATE TABLE Orders (
  CustomerId  STRING(36) NOT NULL,
  OrderId     STRING(36) NOT NULL,
  Status      STRING(32) NOT NULL,
  TotalMinor  INT64 NOT NULL,             -- money in minor units, never a float
  PlacedAt    TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp = true),
) PRIMARY KEY (CustomerId, OrderId),
  INTERLEAVE IN PARENT Customers ON DELETE CASCADE;

CREATE TABLE OrderLines (
  CustomerId  STRING(36) NOT NULL,
  OrderId     STRING(36) NOT NULL,
  LineNo      INT64 NOT NULL,
  Sku         STRING(64) NOT NULL,
  Qty         INT64 NOT NULL,
) PRIMARY KEY (CustomerId, OrderId, LineNo),
  INTERLEAVE IN PARENT Orders ON DELETE CASCADE;

6. Spending consistency deliberately: a stale read for the read-heavy path. The catalog tolerates being a few seconds behind; serving it with a bounded staleness lets any nearby replica answer without a round-trip to the leader.

# Read-heavy, staleness-tolerant path: nearest replica answers, no leader hop.
import datetime
with database.snapshot(exact_staleness=datetime.timedelta(seconds=15)) as snap:
    rows = snap.execute_sql(
        "SELECT Sku, Name, PriceMinor FROM Products WHERE Active = true")
# Cart and checkout stay on STRONG reads (default) — correctness beats latency there.

A few CLI equivalents for readers who reach for gcloud first — the anycast VIP, the Spanner instance, and a region’s Cloud Run deploy:

gcloud compute addresses create app-vip --global --ip-version=IPV4

gcloud spanner instances create app-prod \
  --config=nam-eur-asia1 \
  --description="Global app DB" \
  --processing-units=1000

gcloud run deploy app \
  --region=europe-west1 \
  --image=europe-west1-docker.pkg.dev/PROJECT_ID/app/web:1.0.0 \
  --ingress=internal-and-cloud-load-balancing \
  --service-account=run-app@PROJECT_ID.iam.gserviceaccount.com \
  --min-instances=2 --max-instances=100 --concurrency=80 \
  --no-allow-unauthenticated

Read those three commands against the three Terraform blocks above and the mapping is one-to-one — the CLI is just the imperative view of the same resources.

Enterprise considerations

Security and Zero Trust. The design is defense-in-depth with a single hardened ingress. Cloud Armor provides WAF and DDoS at the edge; the global LB terminates TLS with modern ciphers and HTTP/3; Cloud Run/GKE accept traffic only from the load balancer. Inside the perimeter, VPC Service Controls draw a boundary around Spanner, Secret Manager, and GCS so data cannot egress to an untrusted project. Service-to-service calls are authenticated with service-account identity (and mTLS via Anthos Service Mesh in the GKE variant). Customer data in Spanner is encrypted at rest (optionally with CMEK in Cloud KMS for regulated workloads), and IAP plus context-aware access enforce identity- and device-based access on administrative surfaces — the core Zero-Trust posture: never trust the network, always verify identity.

Cost optimization. This architecture is economical because it is demand-shaped. Cloud Run scales to zero, so non-production environments and low-traffic regions cost almost nothing when idle; you pay per request and per 100 ms of CPU. Cloud CDN offloads the majority of bytes to the edge, cutting both egress and compute. The two line items to watch are Spanner and inter-region egress. Spanner multi-region is the floor cost of the design — start at a small node/PU count and turn on the Spanner autoscaler to track load; use committed-use discounts once your baseline is known. Trim egress with aggressive CDN caching and by keeping chatty service-to-service traffic in-region. A practical rule: start single-region Spanner + multi-region Cloud Run if your RPO tolerance allows, and graduate to multi-region Spanner only when zero-RPO survivability justifies the premium.

Scalability. Each tier scales independently and horizontally. The global LB is effectively unbounded. Cloud Run scales out per region to its max-instances ceiling; raise the ceiling and add regions to grow. Spanner scales by adding processing units with no downtime and no re-sharding — this is its signature property and the reason it anchors the design. Push read-heavy workloads onto stale reads (e.g. 10–15 s staleness) to serve from the nearest replica without a leader round-trip, dramatically increasing read throughput and lowering latency for feeds, catalogs, and dashboards.

Reliability and DR (RTO/RPO). Multi-region Spanner offers a 99.999% availability SLA and, critically, RPO = 0: synchronous Paxos replication means a committed write survives the total loss of any single region with no data loss. RTO is effectively zero for a regional failure — Spanner transparently elects a new leader, and the global LB drains the failed region’s Cloud Run within health-check intervals (seconds), so traffic reroutes automatically without human action. There is no failover runbook to execute for the common case. For correctness and corruption recovery, enable Spanner backups plus point-in-time recovery (PITR) (up to 7 days) to recover from a bad deploy or logical error, and run regular DR game-days that kill a region in staging to validate the automatic behavior is real.

Observability. Instrument the full path. The LB exports request/latency/error metrics per backend and per region; Spanner exposes CPU utilization, latency percentiles, and lock/abort stats; Cloud Run exposes request, instance-count, and cold-start metrics. Define SLOs in Cloud Monitoring (e.g. 99.9% of API requests < 300 ms, served from the user’s region) and alert on burn rate, not raw thresholds. Propagate trace context from edge through Cloud Run to Spanner with Cloud Trace / OpenTelemetry so a slow checkout can be attributed to a specific span — network, cold start, or a hot Spanner key. Use Error Reporting and Profiler to close the loop in production.

Governance. Enforce the perimeter with Organization Policy constraints (block public IPs on VMs, restrict resource locations to approved regions for data residency, require CMEK where mandated). Use folders and projects to separate prod/non-prod and to scope IAM blast radius. Centralize findings in Security Command Center. Tag resources with labels for cost allocation and FinOps reporting, and gate every infra change through PR review on the Terraform repo with policy-as-code (OPA/gcloud policy validator) in CI.

Reference enterprise example

Meridian Threads is a fictional direct-to-consumer apparel brand. Founded in Bengaluru, it now sells in 38 countries with three demand centers: India/SE Asia, Western Europe, and North America. At Series C, the engineering team of 22 hit a wall: their single-region stack in asia-south1 gave a great experience in Mumbai and a sluggish one everywhere else, and a 40-minute regional networking incident during a flash sale cost them an estimated ₹1.1 crore (~$130k) in abandoned carts. The board mandated a “no global-blast-radius” customer tier with a defined RTO/RPO. They adopted this architecture.

What they built. Cloud Run services in asia-southeast1, europe-west1, and us-central1, all behind one Global External Application Load Balancer on a single anycast IP. Cloud CDN fronts the storefront — product images, the Next.js static bundle, and category pages — and absorbs ~88% of total bytes at the edge. Cloud Armor runs OWASP rules plus a 600-requests-per-minute per-IP rate-ban that quietly defeats the credential-stuffing they used to fight manually. The catalog, cart, orders, and inventory live in a single Spanner multi-region instance (nam-eur-asia1), starting at a modest processing-unit count with the autoscaler enabled. Order-confirmation emails, search re-indexing, and the data-warehouse feed go through Pub/Sub to Cloud Run workers, keeping checkout latency clean.

Decisions and trade-offs they made.

• They debated regional sharding to save money and rejected it: a meaningful slice of orders involve customers who travel or have addresses in multiple regions, and the sharding logic would have slowed every future feature. One global Spanner database removed the question entirely.
• They set min-instances=2 only in asia-southeast1 and europe-west1 (their traffic centers) to kill cold starts where it matters, and left us-central1 scaling from zero off-peak — a deliberate latency-vs-cost trade.
• They used 15-second stale reads for the product catalog and bestseller feeds (read-heavy, tolerant of slight staleness) while keeping cart and checkout on strong reads. This roughly tripled catalog read throughput per processing unit and cut catalog latency for distant users.
• They put CMEK on Spanner because they store customer addresses and partial order history, satisfying their EU customers’ data-handling expectations, and pinned non-EU PII processing to approved regions via Org Policy.

The outcome. Median first-contentful-paint for European and SE-Asian shoppers dropped from ~1.9 s to ~520 ms, and checkout p95 latency fell by more than half outside India. Six weeks after cutover, GCP took a real outage in one of their three regions during business hours; the global LB drained that region within health-check intervals and Spanner never lost a write. Customer impact: none. Pages served: every one. Orders lost: zero. The on-call engineer found out from the monitoring channel, not from customers. Conversion in the EU and US markets rose enough in the following quarter that the multi-region Spanner premium — their largest new line item — paid for itself several times over. The 22-person team operates this global footprint without a dedicated DBA, because there is no sharding to babysit and no failover runbook to rehearse for the common case.

When to use it

Use this architecture when you have a genuinely global (or fast-globalizing) user base, your data needs strong consistency (anything touching money, inventory, or identity), and the business has set an aggressive RTO/RPO for the customer-facing tier. It shines for e-commerce, global SaaS, media/subscription, marketplaces, and any “single brand, every continent” product. The scale-to-zero compute and pay-per-use edge mean a small company can adopt the shape early and grow into it, which is the whole point of a reference architecture.

Be honest about the trade-offs. The dominant one is Spanner’s cost and model. Multi-region Spanner has a real monthly floor that a hobby project cannot justify, and Spanner is a (mostly) relational system with its own idioms — interleaving, key design, the absence of some PostgreSQL niceties (though the PostgreSQL interface narrows this gap considerably). If your data is naturally a single-region workload, or you can tolerate an RPO measured in seconds, a regional Cloud SQL (with cross-region read replicas and a documented failover) is far cheaper and simpler — use it and revisit Spanner when global write-survivability actually becomes a requirement. Likewise, if your application is read-mostly with no transactional writes, you may not need Spanner at all; a CDN over a regional database, or AlloyDB for Postgres-heavy analytical-transactional workloads, can be the better fit.

Anti-patterns to avoid. Do not build a load balancer per region and glue them with DNS — you lose anycast entry and instant failover, and you inherit DNS TTL as your RTO floor. Do not put a monotonic primary key on a high-write Spanner table; you will hotspot a single split and conclude, wrongly, that the database is slow. Do not route static assets through Cloud Run — that is what Cloud CDN is for, and skipping it inflates both cost and latency. Do not ship breaking schema migrations in lockstep with multi-region code rollouts; always expand-then-contract. And do not grant services the default compute service account or databaseAdmin — least privilege per service is the baseline, not an enhancement.

Alternatives at the edges. For workloads that need rich service-mesh features, sidecars, or long-lived gRPC streams, swap Cloud Run for the GKE Autopilot variant described above — same global LB, same Spanner, different compute substrate. For a simpler, smaller global app where eventual consistency is acceptable, Firestore in multi-region mode behind the same front end is a lighter-weight data tier. And if you are multi-cloud or portability-constrained, the compute and edge tiers map cleanly to other providers — but Spanner’s globally consistent, horizontally scalable write capability is the piece that is genuinely hard to replicate, and it is usually the reason teams choose GCP for this pattern in the first place.

Going deeper

This is the layer under the diagram — the mechanics an experienced engineer needs to reason about failure, latency, and cost with confidence.

How one IP serves the planet: anycast, GFEs, and the backbone

The “single global anycast IP” is not a metaphor. Google advertises that address via BGP from 100+ points of presence simultaneously; the internet’s routing fabric delivers each user’s packets to the topologically nearest PoP. There, a Google Front End (GFE) terminates TLS. With the global external Application Load Balancer (the EXTERNAL_MANAGED, Envoy-based data plane), you also get HTTP/3 (QUIC) with 0-RTT resumption for returning clients, which shaves a full round-trip off connection setup — meaningful when the user is 200 ms away. Crucially, the user-to-edge hop rides the public internet only as far as the nearest PoP; from the GFE to your regional backend, traffic travels Google’s private backbone, which is faster and less jittery than the open internet. This is why terminating at the edge and back-hauling on Google’s network beats letting a user’s TCP connection stretch all the way to us-central1.

Serverless-NEG steering vs. instance-group balancing (the nuance the Terraform hinted at)

The component table says “outlier detection + health checks for automatic regional drain,” and that is exactly right — for the GKE / managed-instance-group variant. It is worth being precise about how the serverless path differs, because the two behave differently under load:

• Serverless NEGs (Cloud Run / Functions / App Engine): you attach the NEGs to a global backend service and Google routes each request to the closest region, failing over to the next-closest region if the nearest is unavailable. You do not configure balancing_mode, max_rate_per_endpoint, capacity_scaler, health checks, or outlier detection — those fields are rejected on serverless-NEG backends. Google health-checks the serverless platform for you, and “capacity” is elastic because Cloud Run autoscales, so there is no fixed rate to overflow at. Steering is essentially proximity + availability.
• Instance-group / zonal-NEG backends (GKE, MIGs): here you do own capacity. balancing_mode = "RATE" with max_rate_per_endpoint (or UTILIZATION), a capacity_scaler, and outlier_detection give you genuine capacity-aware overflow — when a region hits its configured rate, new requests spill to the next region before anything is unhealthy — plus health-check-driven draining. This is strictly more control, and strictly more configuration to get right.

The practical takeaway: on Cloud Run you get proximity-and-failover essentially for free; on GKE you get capacity-shaping but must tune the balancing mode and health checks. Neither is “better” — they are different levers, and picking the compute substrate is really picking which lever you want.

Spanner’s real magic: TrueTime, Paxos, and the consistency menu

Spanner’s superpower is external consistency (linearizability across the entire database, not just one row) at global scale. Two mechanisms make it possible:

• Paxos replication. Data is chopped into splits (key ranges). Each split has a leader replica and follower replicas spread across the config’s regions. A write is not acknowledged until a quorum of replicas has durably logged it. In a multi-region config there are read-write replicas in two regions plus a witness region that votes on the quorum but stores no data — so a full region can vanish and a quorum still exists. That is the physical reason for RPO = 0.
• TrueTime. Google’s data centers carry GPS and atomic clocks, so every server knows the current time within a tiny, bounded uncertainty. Spanner exploits this: at commit it waits out the uncertainty window (a commit-wait of a handful of milliseconds) so that timestamps are globally ordered. That small, deliberate wait is the price of linearizability — and the reason a multi-region write costs you a cross-region quorum round-trip plus commit-wait, typically low tens of milliseconds. In the PACELC framing, Spanner chooses Consistency both under a partition and in normal operation, paying for it in Latency; it never silently serves you a stale-yet-labelled-fresh value.

This is where the consistency menu earns its keep. You are not stuck paying the leader-round-trip tax on every read:

The “tripled catalog throughput” in the Meridian example is not marketing — it is the direct consequence of moving read-heavy, staleness-tolerant queries off the leader and onto whichever replica is closest to the user.

Cost math, made concrete (representative — always confirm on the pricing calculator)

Order-of-magnitude, so you can sanity-check a proposal before it reaches finance. Rates change and vary by config, so treat these as shapes, not quotes:

• Multi-region Spanner (nam-eur-asia1, ~1 node / 1,000 PU) is a four-figure-USD monthly line item before storage and egress — it is the floor cost of the whole design, and the number that most often surprises a first-time adopter.
• Regional Spanner (single-region, 1 node) is well under half of that for the same PU — the reason “regional Spanner + multi-region Cloud Run” is a legitimate starter posture when RPO-seconds is acceptable.
• Cloud SQL (a small HA regional instance) is another one to two orders of magnitude cheaper again — which is exactly why the “When to use it” section tells you not to reach for Spanner until global write-survivability is a real requirement.
• Cloud Run is per-request + per-100 ms-vCPU; scale-to-zero means idle regions and non-prod cost near nothing. The catch: min-instances are billed even while idle (that is what you are buying — no cold starts), so set them only where latency revenue justifies it.
• Cloud CDN turns egress down: bytes served from cache are cheaper than origin egress and skip compute entirely. The Meridian “88% of bytes at the edge” is the single biggest cost lever in the whole stack.
• Inter-region egress is the quiet tax — chatty service-to-service calls that cross regions add up. Keep the synchronous path in-region and let Pub/Sub absorb the fan-out.

Quotas, limits, and the caveats that bite in production

• Cloud Run default max is 100 instances per service (raisable via quota) and up to 1,000 concurrent requests per instance; concurrency is a throughput/latency dial, not a free win — higher concurrency means more requests sharing one instance’s CPU.
• Spanner scales in processing units (100 PU = 0.1 node); below 1,000 PU you move in increments of 100, above 1,000 in increments of 1,000. Keep node CPU below ~65% (regional) / ~45% (multi-region, because of cross-region work) for headroom; the autoscaler targets exactly this.
• Cloud Armor policies cap the number of rules per policy, and each preconfigured WAF rule has a sensitivity you should tune from preview to enforce gradually to avoid false positives blocking real users.
• Managed certificates cover a bounded number of domains each and take minutes-to-provision on first apply — never assume the cert is live the instant terraform apply returns.
• VPC Service Controls perimeters are powerful and blunt: a misconfigured perimeter blocks your own CI or a legitimate cross-project call just as effectively as an attacker. Roll them out in dry-run mode first and read the violation logs before enforcing — see the VPC Service Controls lesson for the exfiltration model in full.

Schema migrations on one global database

Because all regions and all Cloud Run revisions share one Spanner database, a migration is a distributed-systems problem, not a maintenance window. Spanner DDL is online and non-blocking — adding a column or an index does not lock the table — but an index backfill on a large table is a long-running background operation you must let finish before you depend on it. The discipline is expand → migrate → contract, always across separate releases: (1) add the new column/table (nullable, tolerated by old code); (2) deploy code that writes both shapes and reads the new one, then backfill; (3) once every region runs the new code and the backfill is done, drop the old shape. Ship a breaking DDL in lockstep with a multi-region rollout and you will, for a window, have one revision writing the old shape and another reading the new one against the same rows — a data-integrity incident that is entirely avoidable.

Practice challenges

Work these top to bottom; each builds on the last. Try it yourself before opening the solution.

Challenge 1 — Reserve the front door (Beginner). Reserve a single global anycast IP for the load balancer, and explain in one sentence why one address can serve users on every continent.

<details> <summary>Show solution</summary>

gcloud compute addresses create app-vip --global --ip-version=IPV4

Why: the address is advertised via BGP from every Google PoP simultaneously (anycast), so the internet routes each user to the nearest PoP holding that IP — one address, many physical entry points.

</details>

Challenge 2 — Wire three regions into one backend (Beginner). Given regions europe-west1, us-central1, asia-southeast1, write the Terraform for the per-region serverless NEGs, and state why you must not add a health check to the backend service.

<details> <summary>Show solution</summary>

resource "google_compute_region_network_endpoint_group" "run_neg" {
  for_each              = toset(["europe-west1", "us-central1", "asia-southeast1"])
  name                  = "run-neg-${each.value}"
  region                = each.value
  network_endpoint_type = "SERVERLESS"
  cloud_run { service = google_cloud_run_v2_service.app[each.value].name }
}

Why: serverless-NEG backends reject health_checks and balancing_mode — Google health-checks the Cloud Run platform internally and routes to the closest healthy region for you. Adding a health check is a validation error, not just redundant.

</details>

Challenge 3 — Design a hotspot-proof key (Intermediate). You have a high-write Events table (thousands of inserts/second). Write DDL whose primary key spreads writes evenly across splits, and name the anti-pattern you are avoiding.

<details> <summary>Show solution</summary>

CREATE TABLE Events (
  EventId    STRING(36) NOT NULL,   -- UUIDv4: random prefix distributes writes
  OccurredAt TIMESTAMP NOT NULL OPTIONS (allow_commit_timestamp = true),
  Kind       STRING(64) NOT NULL,
  Payload    JSON,
) PRIMARY KEY (EventId);

Why: the anti-pattern is a monotonic key (auto-increment or a leading timestamp), which funnels every new write to the last split — a single-server hotspot. A UUID (or a hashed/bit-reversed prefix) randomizes the leading bytes so writes fan out across splits.

</details>

Challenge 4 — Defend the login path (Intermediate). Add a Cloud Armor rule set that blocks SQL-injection attempts and rate-bans any IP exceeding 100 requests/minute. Give the two rules (plus the mandatory default).

<details> <summary>Show solution</summary>

rule {                                   # block SQLi
  action = "deny(403)"  priority = 1000
  match { expr { expression = "evaluatePreconfiguredExpr('sqli-v33-stable')" } }
}
rule {                                   # 100 req/min per IP, then ban
  action = "rate_based_ban"  priority = 1100
  match { versioned_expr = "SRC_IPS_V1"  config { src_ip_ranges = ["*"] } }
  rate_limit_options {
    enforce_on_key = "IP"  conform_action = "allow"  exceed_action = "deny(429)"
    ban_duration_sec = 600
    rate_limit_threshold { count = 100  interval_sec = 60 }
  }
}
rule {                                   # required catch-all default
  action = "allow"  priority = 2147483647
  match { versioned_expr = "SRC_IPS_V1"  config { src_ip_ranges = ["*"] } }
}

Why: the preconfigured sqli-v33-stable expression is Google’s maintained OWASP SQLi rule set, and rate_based_ban enforces a per-IP threshold with a cooling-off ban — far cheaper and faster than fighting credential-stuffing in application code. Roll the WAF rule out in preview first to catch false positives.

</details>

Challenge 5 — Fix a hot read path without adding capacity (Advanced). A /catalog endpoint is pinning the Spanner leader and is slow for users far from the leader region. Reduce leader load and tail latency without adding processing units or replicas.

<details> <summary>Show solution</summary>

import datetime
# Catalog tolerates ~15s staleness -> serve from the nearest replica, no leader hop.
with database.snapshot(exact_staleness=datetime.timedelta(seconds=15)) as snap:
    rows = snap.execute_sql("SELECT Sku, Name, PriceMinor FROM Products WHERE Active")

Why: a strong read must consult the leader; an exact-staleness read is answered by whatever replica is closest and recent enough, removing the cross-region round-trip and offloading the leader. Keep cart/checkout on strong reads — spend staleness only where correctness allows.

</details>

Challenge 6 — Ship a column rename across three regions safely (Advanced). Code in all three regions reads Orders.Status. Product wants it renamed to FulfillmentState. Give the release sequence that never breaks a live region.

<details> <summary>Show solution</summary>

Use expand → migrate → contract, one release per step:

1. Expand: ALTER TABLE Orders ADD COLUMN FulfillmentState STRING(32) (nullable; old code ignores it).
2. Migrate: deploy code that writes both columns and reads the new one; backfill FulfillmentState from Status for existing rows.
3. Contract: once every region runs the new revision and the backfill is complete, ALTER TABLE Orders DROP COLUMN Status.

Why: the single global database is shared by every region and revision simultaneously, so at no instant can the schema be incompatible with any running code. Shipping the rename as one breaking DDL would leave one revision writing Status while another reads FulfillmentState against the same rows — a data-integrity incident.

</details>

Common beginner mistakes

• “I’ll put a load balancer in each region and point DNS at them.” The misconception is that DNS-based geo-routing equals a global front door. It does not: you lose anycast entry, and DNS TTL becomes your RTO floor because clients and resolvers cache the record long after a region dies. Right model: one global backend service with regional serverless NEGs — failover happens inside the LB in seconds, and every user shares one anycast IP.

• “An auto-increment or timestamp primary key is fine for Spanner.” It looks fine at low volume and falls over under write load. Monotonic keys send every new row to the same last split — a single-server hotspot — and the benchmark gets blamed on “Spanner being slow.” Right model: high-write tables use UUIDs, hashed prefixes, or bit-reversed sequences so writes fan out across splits.

• “Serve static assets from Cloud Run — it’s simpler.” You then pay compute and origin egress for bytes that a CDN serves free from the edge, and the user waits for a region round-trip on every image. Right model: static paths go to a CDN-enabled bucket backend; only truly dynamic paths reach Cloud Run.

• “Strong reads everywhere is the safe default.” Correctness is not free: every strong read consults the leader, so you pay a cross-region round-trip and cap read throughput at the leader’s capacity — overkill for catalogs and feeds. Right model: strong reads for money/inventory/identity, bounded or exact staleness for read-heavy, staleness-tolerant paths.

• “I’ll ship the schema change with the code that needs it.” On one shared global database, a mid-rollout window has both code versions live against the same rows. A breaking DDL in that window is a data-integrity incident. Right model: expand-then-contract across separate releases, always backward-compatible.

• “Short DNS TTLs are my failover mechanism.” Failover in this design lives in the load balancer (health-check intervals, seconds), not in DNS. Chasing failover with tiny TTLs just hammers your DNS and still leaves cached records stale. Right model: anycast + LB draining does the failover; DNS stays boring.

• “Use the default compute service account — it already works.” The default SA is broadly privileged and shared, so one compromised service can reach far more than it should, and databaseAdmin lets a web tier alter schema. Right model: one dedicated least-privilege SA per service, with roles/spanner.databaseUser on the specific database and secret-accessor on the specific secrets.

• “We’re global, so multi-region Spanner from day one.” Multi-region Spanner has a real four-figure monthly floor that a pre-revenue product cannot justify. Right model: adopt the shape (multi-region Cloud Run + CDN), and start the data tier at the survivability you actually need — regional Spanner or even Cloud SQL — graduating to multi-region Spanner when zero-RPO across regions becomes a genuine requirement.

Glossary

• Anycast IP — one IP address advertised from many locations at once; the network delivers each user to the nearest one. Here, the single VIP that fronts the whole app worldwide.
• Global External Application Load Balancer — a single global L7 load balancer (the EXTERNAL_MANAGED, Envoy-based variant) that terminates TLS at the edge and routes to regional backends.
• VIP — Virtual IP; the stable front-end IP address of the load balancer.
• Serverless NEG — a network endpoint group that points a load balancer backend at a serverless service (Cloud Run/Functions/App Engine) rather than at VMs.
• Backend service — the load balancer object that groups backends and decides how to route to them. In this design it is global and singular — the seam of the architecture.
• URL map — the routing table inside the load balancer that maps paths/hosts to backends (static → CDN bucket, dynamic → Cloud Run).
• Cloud CDN — Google’s content delivery network; caches static and cacheable content at edge PoPs so bytes never reach a region.
• PoP (point of presence) / edge — a Google network location close to users where TLS terminates and cache hits are served.
• Cloud Armor — the edge WAF and DDoS defense; runs OWASP rule sets, rate-based bans, and adaptive protection before requests reach compute.
• WAF — Web Application Firewall; filters malicious requests (SQLi, XSS, etc.) by pattern.
• Rate-based ban — a Cloud Armor action that blocks an IP for a cooling-off period once it exceeds a request threshold.
• Cloud Run — fully managed, stateless, autoscaling container platform that scales to zero; the primary compute tier here.
• Cold start — the extra latency when a request must spin up a new instance because none was warm; mitigated with min-instances.
• min-instances / max-instances — the warm-instance floor (billed, kills cold starts) and the scaling ceiling (cost cap) for a Cloud Run service.
• Concurrency — how many requests one Cloud Run instance handles at once; a throughput-vs-latency dial.
• GKE Autopilot — Google’s managed Kubernetes mode; the compute variant when you need sidecars, mesh, gRPC streaming, or stateful workloads.
• Container-native / zonal NEG — a backend that targets pod/endpoint IPs directly (the GKE path), supporting balancing modes and health checks.
• Balancing mode — on instance-group/zonal-NEG backends, the rule (RATE/UTILIZATION) that defines a region’s capacity before traffic overflows to the next region.
• Cloud Spanner — Google’s globally distributed, strongly consistent, horizontally scalable relational database; the keystone that eliminates sharding.
• Multi-region config — a Spanner instance configuration (e.g. eur6, nam-eur-asia1) that synchronously replicates across regions.
• Processing unit (PU) / node — the unit of Spanner capacity; 1,000 PU = 1 node. Scale up/down with no downtime.
• Split — a contiguous primary-key range that Spanner places on a server; the unit it scales and balances by.
• Hotspot — all traffic hitting one split (usually from a monotonic key), starving the rest of the instance.
• Interleaving — physically co-locating child rows under their parent (e.g. order lines under an order) so related data and joins stay on one split.
• Paxos — the consensus protocol Spanner uses to commit a write only after a quorum of replicas agrees — the basis of RPO = 0.
• TrueTime — Google’s globally synchronized clock with bounded uncertainty; lets Spanner order transactions globally via a small commit-wait.
• External consistency — Spanner’s guarantee that transactions appear in a single global order (linearizability) across the whole database.
• Strong read — a read guaranteed to see the latest committed data; consults the leader.
• Bounded / exact staleness — reads allowed to be slightly behind, served by the nearest replica without a leader round-trip; the key to cheap read scaling.
• RPO (Recovery Point Objective) — how much data you can afford to lose. Multi-region Spanner delivers RPO = 0.
• RTO (Recovery Time Objective) — how quickly you must recover. Here it is effectively zero for a regional loss — the LB drains and Spanner re-elects automatically.
• PITR (point-in-time recovery) — restoring Spanner to a moment in the recent past (up to 7 days) to undo a bad deploy or logical error.
• Direct VPC egress — Cloud Run’s connector-less path onto the VPC; lower latency than the legacy Serverless VPC Access connector.
• Cloud NAT — managed NAT that gives serverless/private workloads stable, allowlistable egress IPs for third-party APIs.
• VPC Service Controls — a data-exfiltration boundary around services (Spanner, GCS, Secret Manager) so data cannot leave the trusted perimeter even if a credential leaks.
• Secret Manager — central, access-controlled store for credentials and keys, read at runtime by a service’s SA — never baked into images or env vars.
• Workload Identity Federation — keyless authentication (for GKE workloads and CI) that removes long-lived JSON service-account keys entirely.
• IAP (Identity-Aware Proxy) — an identity- and context-aware gate on admin/internal surfaces; core to the Zero-Trust posture.
• CMEK — Customer-Managed Encryption Keys in Cloud KMS; you control the key that encrypts data at rest, for regulated workloads.
• Pub/Sub — managed messaging that decouples async work (email, indexing, analytics) from the synchronous request path.
• Expand-then-contract — the safe migration pattern for a shared database: add the new shape, run both, backfill, then remove the old shape — each in its own release.
• HTTP/3 (QUIC) — the UDP-based transport the global ALB offers at the edge; 0-RTT resumption cuts connection-setup latency for distant users.
• Google-managed certificate — a TLS certificate Google provisions and auto-renews for the load balancer’s hostnames.
• SLO / burn rate — a Service Level Objective (e.g. 99.9% of requests < 300 ms) and the rate at which you are consuming its error budget; alert on burn rate, not raw thresholds.