Implementing Data Mesh: Domain Data Products and Federated Computational Governance

Data mesh fails in practice for a reason that has nothing to do with technology: organizations adopt the org chart and skip the engineering. They rename the central data team to “platform,” tell every business unit they now “own their data,” and declare victory. Six months later there are forty Snowflake schemas, no two of which agree on what a customer_id is, and the only person who understands the lineage just left. Data mesh is not a reorg. It is a socio-technical architecture where four principles reinforce each other, and removing any one collapses the rest. This article is about operationalizing all four, with the assumption that you have already decided your central warehouse has become the bottleneck Zhamak Dehghani described, and you want the engineering substance, not the manifesto.

The running example is a mid-size retailer with domains for Orders, Inventory, Customer, and Pricing, migrating off a monolithic warehouse that one 30-person data team can no longer keep current.

1. The four principles and the problem each one solves

Dehghani’s four principles are not a menu. Each exists to neutralize a failure mode the previous one introduces.

The critical insight: decentralized ownership without product thinking is just shadow IT, and product thinking without a self-serve platform just makes every team hire a data engineer. The fourth principle is what stops the first three from devolving into chaos. “Computational” is the load-bearing word – governance is encoded as policy that executes automatically in the platform, not as a wiki page a review board points at.

A test I apply: if your “governance” can be ignored by a determined engineer pushing to main on a Friday, you have a policy document, not computational governance.

2. Define a data product: ports, SLAs, and discoverability

A data product is the atomic unit of a mesh. It is not a table. It is a self-contained, independently deployable unit that exposes data through well-defined ports, ships with its own metadata, and meets published service levels. Dehghani’s framing – a data product should be discoverable, addressable, trustworthy, self-describing, interoperable, and secure – becomes a concrete interface contract.

Every data product exposes three classes of port:

• Output ports – how consumers read the data (a SQL view, an Iceberg table, a Kafka topic, an API). A product may expose several output ports over the same logical data.
• Input ports – where the product ingests from (upstream operational DBs, events). Internal; consumers never touch these.
• Control ports – operational interface: trigger a rebuild, query freshness, fetch the contract.

I make every data product self-describe with a single declarative spec, versioned in the domain’s repo. This is the canonical artifact the platform reads to provision infrastructure, register the catalog, and enforce policy.

# orders/data-products/order-events/dataproduct.yaml
apiVersion: dataproduct.kloudvin.io/v1
kind: DataProduct
metadata:
  name: order-events
  domain: orders
  owner: orders-data@retailer.example
spec:
  description: "Authoritative stream of placed, fulfilled, and cancelled orders."
  outputPorts:
    - name: iceberg-curated
      type: iceberg
      location: s3://mesh-orders/order_events/
      contract: ./contracts/order-events.v2.yaml
    - name: kafka-realtime
      type: kafka
      topic: orders.order-events.v2
  sla:
    freshness: "<= 15m"        # max lag from operational system
    availability: "99.5%"
    completeness: ">= 99.9%"   # rows reconciled against source-of-truth count
  classification:
    contains_pii: true
    pii_fields: [customer_id, shipping_address]

Discoverability is non-negotiable: a product nobody can find does not exist. The spec above is what populates the catalog automatically. Addressability means a stable, global identifier – here, orders.order-events – that survives storage migrations. SLAs are not aspirational; they are measured and reported on the control port, and a product that misses them is paged on, not quietly ignored.

3. Domain ownership and the data product team operating model

Ownership only works if the team has end-to-end responsibility – ingestion, transformation, quality, the contract, and the on-call pager. The anti-pattern is “domain owns the business logic, central team owns the pipeline,” which recreates the bottleneck with extra meetings.

A viable data product team is small and cross-functional:

Three rules keep this from collapsing. First, the team that generates the operational data owns the corresponding source-aligned data product – ownership follows where the truth originates, never gets assigned to whoever has spare capacity. Second, consumer-aligned aggregate products (a “customer 360” joining Orders, Customer, and Pricing) get their own owning team; they are not a free externality dumped on the source domains. Third, no domain runs its own bespoke infrastructure – if a team is standing up its own Airflow cluster, the platform has failed them, and that is a platform bug to fix, not a domain decision to celebrate.

4. The self-serve data platform plane and golden paths

The platform’s job is to make the right way the easy way. Its success metric is brutally simple: time for a domain team to ship a new, governed, discoverable data product from zero. If that is weeks, you do not have a platform; you have a ticket queue. Target hours.

Think in planes, not a monolith:

• Infrastructure utility plane – storage (object store + table format like Iceberg/Delta), compute (Spark/Trino/dbt runners), streaming (Kafka), orchestration. Domains never provision raw cloud resources.
• Data product experience plane – the abstraction domains actually touch: dataproduct.yaml, a CLI, templates. They declare what, not how.
• Mesh supervision plane – cross-product concerns: the catalog, global lineage, policy engine, SLA monitoring.

The deliverable is a golden path: a paved, opinionated route that provisions everything correctly by default. A domain engineer runs one command and gets a scaffolded product wired to storage, the catalog, CI, and policy checks.

# Golden path: scaffold a new data product, fully wired
mesh init dataproduct \
  --domain orders \
  --name returns \
  --template source-aligned-iceberg \
  --classification pii

# Generated: dataproduct.yaml, contracts/, dbt model skeleton,
# CI pipeline with contract + policy gates, catalog registration hook

Under the hood the platform ships these templates as infrastructure modules so the generated infra is consistent and reviewable, not hand-rolled per team:

# platform/modules/data-product/main.tf  (consumed by the golden path)
module "data_product_storage" {
  source         = "./storage"
  product_name   = var.product_name
  domain         = var.domain
  table_format   = "iceberg"
  retention_days = var.classification == "pii" ? 365 : 1095
}

resource "aws_glue_catalog_database" "product" {
  name = "${var.domain}_${var.product_name}"
  tags = {
    domain          = var.domain
    contains_pii    = tostring(var.classification == "pii")
    managed_by_mesh = "true"
  }
}

The platform is itself a product. Treat domain teams as customers, run a backlog, and measure adoption. A golden path nobody uses because it is too rigid is as much a failure as no platform at all.

5. Federated computational governance and policy-as-code

This is where most implementations are weakest, because it is the hardest. Federated means a guild of domain representatives plus platform plus security sets global rules; domains decide everything local. Computational means those global rules execute as code in the platform, automatically, with no human gate in the happy path.

The governance body decides the small set of things that must be global to interoperate:

• Polyseme identification – which concepts cross domains (customer_id, sku, currency) and their canonical definition and type.
• Classification taxonomy and the controls each class triggers (PII -> encryption at rest, masked default view, retention ceiling).
• The required shape of every data product (must have a contract, an owner, SLAs, and a passing quality suite).

Encode each decision as policy. With Open Policy Agent, a product cannot register unless it satisfies the global rules – enforced in CI and again at the catalog admission point:

# policy/dataproduct.rego
package mesh.dataproduct

import rego.v1

deny contains msg if {
    input.spec.classification.contains_pii == true
    not input.spec.outputPorts[_].contract
    msg := "PII data products must declare a data contract on every output port"
}

deny contains msg if {
    some port in input.spec.outputPorts
    port.type == "iceberg"
    not endswith(port.location, "/")
    msg := sprintf("output port %v: iceberg location must be a directory prefix", [port.name])
}

deny contains msg if {
    not input.spec.sla.freshness
    msg := "every data product must publish a freshness SLA"
}

# CI gate -- runs on every PR touching a data product
conftest test orders/data-products/returns/dataproduct.yaml \
  --policy policy/ --namespace mesh.dataproduct

The distinction from old-world central governance: nobody reviews the returns product before launch. The policy is the review, it runs in milliseconds, and it is identical for every domain. Humans in the governance guild change the policy; they do not gate deployments.

6. Data contracts, schema evolution, and consumer-driven compatibility

The contract is the API of a data product, and like any API it must evolve without breaking consumers. A contract declares schema, semantic types, quality expectations, and the SLA – machine-readable and version-controlled alongside the product.

# contracts/order-events.v2.yaml
dataContractSpecification: 1.1.0
id: orders.order-events
info:
  version: 2.0.0
  owner: orders-data@retailer.example
models:
  order_event:
    type: table
    fields:
      order_id:    { type: string, required: true, unique: true, primaryKey: true }
      customer_id: { type: string, required: true }   # polyseme: global definition
      status:      { type: string, required: true, enum: [placed, fulfilled, cancelled] }
      total_minor: { type: long,   required: true, description: "Order total in minor units (cents)" }
      currency:    { type: string, required: true, pattern: "^[A-Z]{3}$" }
      occurred_at: { type: timestamp, required: true }
quality:
  - type: row_count
    model: order_event
    mustBeGreaterThan: 0
  - type: sql
    description: "no negative totals"
    query: "SELECT COUNT(*) FROM order_event WHERE total_minor < 0"
    mustBe: 0
servicelevels:
  freshness: { threshold: 15m }

The governing rule for evolution is producers may only make backward-compatible changes silently; anything else is a new major version. Adding an optional field or a new enum value at the end is safe. Removing a field, renaming, narrowing a type, or tightening required is breaking – it ships as v3 on a new output port, and v2 stays alive through a published deprecation window while consumers migrate.

Enforce this consumer-driven. Consumers register the subset of the contract they actually depend on; producer CI runs every registered consumer expectation against the proposed schema and fails the build if any break. This is the same logic as consumer-driven contract testing for service APIs, applied to data:

# Producer CI -- block schema changes that break registered consumers
datacontract changelog contracts/order-events.v2.yaml \
  contracts/order-events.v3-proposed.yaml

# Run quality + schema checks against the live output port
datacontract test contracts/order-events.v2.yaml \
  --server iceberg-curated

Pair this with a schema registry on the streaming output port set to BACKWARD compatibility, so the same guarantee holds for the Kafka port that the contract test holds for the table port – one rule, enforced at every boundary.

7. Interoperability standards, cataloging, and cross-domain joins

The payoff of all the governance machinery is that cross-domain analytics just works. Three standards make products composable.

Global identifiers. Polysemes resolve to one canonical type and meaning. The governance guild decides customer_id is a string of a specific format across Orders, Customer, and Pricing. A join across domains is then a plain join, not a reconciliation project.

A single open table format. Standardize on Iceberg (or Delta) so any compliant engine – Trino, Spark, Snowflake, DuckDB – reads any product without copies. This is what lets a consumer-aligned product join across domains without ETL’ing everything into one warehouse first.

An active catalog as the discovery and governance fabric. The catalog is fed automatically from every dataproduct.yaml, carries lineage and the contract, and is where consumers search. A useful adoption metric is the share of products with a complete, machine-generated catalog entry.

With those in place, a cross-domain query is mundane – which is exactly the goal:

-- Trino federating two independently owned Iceberg data products
SELECT c.segment,
       count(*)                         AS orders,
       sum(o.total_minor) / 100.0       AS gross_revenue
FROM   orders.order_events     AS o
JOIN   customer.customer_profile AS c
       ON o.customer_id = c.customer_id   -- same polyseme, same type, no munging
WHERE  o.status = 'fulfilled'
  AND  o.occurred_at >= current_date - INTERVAL '30' DAY
GROUP  BY c.segment
ORDER  BY gross_revenue DESC;

No central team built this. Two domains published governed products; a consumer joined them. That is the entire promise of the mesh delivered in one query.

8. Incremental adoption and migrating off a central warehouse

Do not big-bang. A mesh migration that pauses delivery to “do it properly” gets cancelled at the first quarterly review. Strangle the warehouse incrementally.

1. Build the thin platform slice first. Storage, table format, catalog, the dataproduct.yaml spec, one golden path, and the OPA policy gate. Resist building the whole platform up front – you do not yet know what domains need.
2. Pick a lighthouse domain with a motivated team and real consumer pain (Orders is ideal: high-value, many downstream users). Ship one source-aligned product end to end. This is your reference implementation and your credibility.
3. Strangle, do not rebuild. Point new consumers at the new product; leave the warehouse table running, fed in parallel, until consumers migrate. Reconcile row counts between old and new continuously.
4. Expand domain by domain, harvesting platform gaps from each into the golden path. Stand up the federated governance guild with the second domain – not before (premature) and not after (chaos has set in).
5. Decommission warehouse tables only when their lineage shows zero remaining consumers, then delete.

-- Reconciliation during the strangle window: old warehouse vs new product
SELECT
  (SELECT count(*) FROM legacy_warehouse.fct_orders
     WHERE order_date = current_date - 1)        AS legacy_count,
  (SELECT count(*) FROM orders.order_events
     WHERE date(occurred_at) = current_date - 1) AS mesh_count;
-- alert if the delta exceeds the completeness SLA threshold

Enterprise scenario

A retail bank with ~25 domain teams ran a single Snowflake warehouse owned by a central team of 40. The constraint that forced the move was regulatory, not capacity: under data-residency rules, EU customer data could not leave the EU region, but the central warehouse was US-based and analysts routinely joined EU customer data into global aggregates. The central team had become both the bottleneck and a compliance liability – every new dataset needed their backlog, and they had no systematic way to prove a given query never moved restricted data across the boundary.

The mesh refactor solved it by making residency a classification attribute on the data product and a computational policy in the platform, rather than a manual review. Each product declared its residency in the spec, and an OPA policy blocked any output port that would place EU-classified data outside an EU-region store. Because the rule executed in CI and at catalog admission, no engineer could accidentally provision a non-compliant product, and the catalog became the audit evidence the regulator wanted.

# policy/residency.rego
package mesh.residency
import rego.v1

deny contains msg if {
    input.spec.classification.residency == "EU"
    some port in input.spec.outputPorts
    not contains(port.location, "eu-")   # require an EU-region prefix in the location
    msg := sprintf(
      "output port %v stores EU-resident data outside the EU region",
      [port.name],
    )
}

The decisive outcome was organizational, not technical: residency stopped being a quarterly audit scramble and became a build-time gate that could not be skipped. New EU-region products shipped in days instead of waiting on a central backlog, and “prove no restricted data crossed the boundary” became a catalog query rather than a forensics project. The bank’s own lesson, which I repeat to every team starting this: lead with the one policy that is causing real pain, encode that computationally first, and let the rest of the mesh follow the credibility it earns.

Verify

Validate the mesh end to end, not piecewise:

# 1. The product spec satisfies global governance policy
conftest test orders/data-products/order-events/dataproduct.yaml \
  --policy policy/ --combine

# 2. The live output port matches its contract (schema + quality)
datacontract test contracts/order-events.v2.yaml --server iceberg-curated

# 3. A proposed schema change breaks no registered consumer
datacontract changelog contracts/order-events.v2.yaml \
  contracts/order-events.v3-proposed.yaml

# 4. The product is discoverable and addressable in the catalog
mesh catalog get orders.order-events --show lineage,contract,sla

# 5. SLA is actually met, not just declared (freshness lag)
mesh sla check orders.order-events --slo freshness

A mesh is healthy when: a new consumer finds and queries a product without talking to its owners; a cross-domain join works on identifier match alone; and a breaking change is caught in CI, not in a consumer’s dashboard.

Implementation checklist

• All four principles are present – ownership, product, platform, and governance reinforce each other rather than one being adopted alone
• Every data product has a versioned dataproduct.yaml with output ports, SLAs, and a classification
• Source-aligned products are owned by the team that generates the operational data; aggregates have their own owning teams
• A self-serve golden path provisions a governed, discoverable product in hours, not weeks
• Time-to-new-product and platform adoption are tracked as platform success metrics
• Global governance is encoded as policy-as-code (OPA/Conftest) and enforced in CI and at catalog admission – no human gate in the happy path
• Polysemes (customer_id, sku, currency) have one canonical, governed definition and type
• Every output port has a data contract; schema changes are validated consumer-driven, with backward-compatible-only silent evolution
• Breaking changes ship as a new major version on a new port with a published deprecation window
• One open table format (Iceberg/Delta) enables cross-domain joins without copies
• The catalog is populated automatically from product specs and carries lineage, contract, and SLA
• Migration strangles the warehouse incrementally behind a lighthouse domain, with continuous reconciliation, never a big-bang cutover