Building an Internal Cloud API with Crossplane Compositions and XRDs

Most platform teams end up writing a Terraform module, wrapping it in a CI pipeline, and calling that “self-service.” It isn’t. Application teams still file tickets, wait on plan approvals, and have no live representation of what they own. Crossplane flips the model: you define an API in your Kubernetes cluster, application teams kubectl apply a small claim, and a reconciler continuously drives cloud resources to match. The platform team ships a versioned API; consumers never see a provider credential or a *.tf file.

This guide stands up a control plane, designs a XPostgreSQLInstance abstraction, wires it to AWS RDS through a Composition, and hardens it for multi-tenant, GitOps-driven delivery. Everything targets Crossplane v1.15+ with the new function-based composition pipeline, which is the path forward now that native patch-and-transform has moved into a function itself.

1. Install Crossplane and configure a provider with controller identity

Crossplane is a set of controllers plus a handful of CRDs. Install it with the official Helm chart into its own namespace.

helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

helm install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system \
  --create-namespace \
  --version 1.18.0 \
  --wait

kubectl get pods -n crossplane-system

Crossplane core does nothing on its own. You install a Provider package to get managed resources for a cloud. The modern AWS provider is split into family packages, so you only pull the controllers you need. Install the RDS family:

# provider-aws-rds.yaml
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-rds
spec:
  package: xpkg.upbound.io/upbound/provider-aws-rds:v1.21.0

kubectl apply -f provider-aws-rds.yaml
kubectl get providers
kubectl wait provider.pkg/provider-aws-rds --for=condition=Healthy --timeout=300s

Controller identity, not static keys

Do not feed the provider a long-lived access key. On EKS, attach an IAM role to the provider’s controller ServiceAccount via IRSA (or EKS Pod Identity). The provider controller runs as a per-provider ServiceAccount that you target with a DeploymentRuntimeConfig:

# runtime-config.yaml
apiVersion: pkg.crossplane.io/v1beta1
kind: DeploymentRuntimeConfig
metadata:
  name: irsa-runtime
spec:
  serviceAccountTemplate:
    metadata:
      annotations:
        eks.amazonaws.com/role-arn: arn:aws:iam::111122223333:role/crossplane-rds-provider
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws-rds
spec:
  package: xpkg.upbound.io/upbound/provider-aws-rds:v1.21.0
  runtimeConfigRef:
    name: irsa-runtime

Then tell the provider to source credentials from the pod’s environment rather than a Kubernetes secret:

# providerconfig.yaml
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  credentials:
    source: IRSA

The blast radius of this ServiceAccount is your entire RDS estate. Scope the IAM policy to the exact actions the provider needs (rds:* on tagged resources), and treat the control plane cluster as Tier-0 infrastructure with its own hardened access path.

2. Managed Resources vs Composite Resources: the reconciliation model

Two reconciliation layers stack here, and conflating them is the most common source of confusion.

A Managed Resource (MR) is a 1:1 Kubernetes representation of one external cloud resource: one Instance.rds.aws.upbound.io maps to one RDS DB instance. Its provider controller runs an external-name-keyed reconcile loop: observe the cloud, diff against spec.forProvider, and call the cloud API to converge. This is where drift correction lives. If someone resizes the instance in the AWS console, the MR controller reverts it on the next reconcile.

A Composite Resource (XR) is a higher-level object you define. It has no cloud controller of its own. Instead, Crossplane’s composition engine reconciles it by rendering a set of MRs from a Composition, applying them, and propagating their status back up. The XR is the unit of abstraction; the MRs are the implementation.

The mental model: MRs are the assembly language of your cloud; XRs are the functions you expose. You are about to define the function signature.

3. Design an XRD that exposes a clean platform-team API

A CompositeResourceDefinition (XRD) defines the schema and identity of your XR. This is the contract. Spend your design effort here, because every consumer and every Composition depends on it, and breaking it later is a versioned migration.

# xrd-postgres.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xpostgresqlinstances.platform.acme.io
spec:
  group: platform.acme.io
  names:
    kind: XPostgreSQLInstance
    plural: xpostgresqlinstances
  claimNames:
    kind: PostgreSQLInstance
    plural: postgresqlinstances
  defaultCompositionRef:
    name: xpostgres-aws
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                parameters:
                  type: object
                  properties:
                    storageGB:
                      type: integer
                      minimum: 20
                      maximum: 1000
                    size:
                      type: string
                      enum: ["small", "medium", "large"]
                    version:
                      type: string
                      default: "16"
                  required:
                    - size
              required:
                - parameters
            status:
              type: object
              properties:
                endpoint:
                  type: string
                  description: Connection hostname for the database.

Design notes that separate a good XRD from a leaky one:

• Model intent, not implementation. Expose size: small|medium|large, not dbInstanceClass: db.t3.medium. The instance class is a Composition decision; if you change the t-shirt-to-class mapping later, no consumer changes.
• Constrain at the schema. minimum, maximum, and enum are validated by the API server at admission. Bad input is rejected before any cloud call.
• referenceable: true marks the version a Composition is allowed to bind to. defaultCompositionRef lets a claim omit composition selection entirely.
• Status is your output contract. status.endpoint is what consumers read back. Keep it stable.

Apply it, and Crossplane generates the XR CRD plus, because you set claimNames, a namespaced claim CRD:

kubectl apply -f xrd-postgres.yaml
kubectl get xrd xpostgresqlinstances.platform.acme.io
kubectl get crd | grep platform.acme.io

4. Author a Composition with the function pipeline, patches, and connection secrets

A Composition tells Crossplane how to render MRs for one XR. As of v1.14+, Compositions run a pipeline of functions rather than the legacy inline resources array. The patch-and-transform behavior everyone knows now lives in function-patch-and-transform. Install it first:

# function-pnt.yaml
apiVersion: pkg.crossplane.io/v1
kind: Function
metadata:
  name: function-patch-and-transform
spec:
  package: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform:v0.7.0

Now the Composition. It maps the abstract size to a concrete instance class, wires storage, and exposes the database endpoint as a connection detail.

# composition-aws.yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: xpostgres-aws
spec:
  compositeTypeRef:
    apiVersion: platform.acme.io/v1alpha1
    kind: XPostgreSQLInstance
  mode: Pipeline
  pipeline:
    - step: patch-and-transform
      functionRef:
        name: function-patch-and-transform
      input:
        apiVersion: pt.fn.crossplane.io/v1beta1
        kind: Resources
        resources:
          - name: rds-instance
            base:
              apiVersion: rds.aws.upbound.io/v1beta1
              kind: Instance
              spec:
                forProvider:
                  region: us-east-1
                  engine: postgres
                  publiclyAccessible: false
                  skipFinalSnapshot: true
                  autoGeneratePassword: true
                  passwordSecretRef:
                    namespace: crossplane-system
                    name: rds-creds
                    key: password
                  username: masteruser
                writeConnectionSecretToRef:
                  namespace: crossplane-system
            connectionDetails:
              - name: endpoint
                type: FromFieldPath
                fromFieldPath: status.atProvider.address
              - name: port
                type: FromFieldPath
                fromFieldPath: status.atProvider.port
            patches:
              - type: FromCompositeFieldPath
                fromFieldPath: spec.parameters.version
                toFieldPath: spec.forProvider.engineVersion
              - type: FromCompositeFieldPath
                fromFieldPath: spec.parameters.storageGB
                toFieldPath: spec.forProvider.allocatedStorage
              - type: FromCompositeFieldPath
                fromFieldPath: spec.parameters.size
                toFieldPath: spec.forProvider.instanceClass
                transforms:
                  - type: map
                    map:
                      small: db.t3.medium
                      medium: db.r6g.large
                      large: db.r6g.2xlarge
              - type: ToCompositeFieldPath
                fromFieldPath: status.atProvider.address
                toFieldPath: status.endpoint
              - type: FromCompositeFieldPath
                fromFieldPath: metadata.uid
                toFieldPath: spec.forProvider.tags["crossplane-uid"]

Three patterns worth internalizing:

1. Direction matters. FromCompositeFieldPath reads the XR and writes the MR (input flow). ToCompositeFieldPath reads the MR’s observed status and writes the XR status (output flow). The endpoint round-trips up through the latter.
2. Transforms are pure functions on a value. The map transform turns t-shirt sizes into instance classes inside the pipeline. No conditionals leak to the consumer.
3. Connection details aggregate up. writeConnectionSecretToRef on the MR plus the connectionDetails block makes Crossplane publish a secret next to the XR (and, with a claim, copy it into the consumer’s namespace). The application gets endpoint, port, and password as a single secret it can mount.

5. Composition Functions for logic beyond patch-and-transform

Patch-and-transform is declarative, which is its strength and its ceiling. The moment you need a loop (fan out N subnets), a conditional resource (create a replica only for large), or cross-resource arithmetic, write a Composition Function. Functions are gRPC services Crossplane calls in the pipeline; they receive the observed state and return a desired state.

You can write them in Go with function-sdk-go, but for most platform logic KCL via function-kcl is faster to ship and reads cleanly. The KCL function takes the observed XR and emits desired MRs:

# main.k  (KCL function logic, conditional replica)
oxr = option("params").oxr  # observed composite resource
size = oxr.spec.parameters.size

_items = [
    {
        apiVersion = "rds.aws.upbound.io/v1beta1"
        kind = "Instance"
        metadata.name = oxr.metadata.name + "-primary"
        spec.forProvider = {
            region = "us-east-1"
            engine = "postgres"
            instanceClass = "db.r6g.2xlarge" if size == "large" else "db.t3.medium"
        }
    }
]

# Only large tiers get a read replica.
if size == "large":
    _items += [{
        apiVersion = "rds.aws.upbound.io/v1beta1"
        kind = "Instance"
        metadata.name = oxr.metadata.name + "-replica"
        spec.forProvider.replicateSourceDb = oxr.metadata.name + "-primary"
    }]

items = _items

Wire it as an additional pipeline step. Steps run in order, and later steps see the desired state produced by earlier ones, so you can layer function-kcl for logic and function-patch-and-transform for field plumbing in the same Composition:

  pipeline:
    - step: render-resources
      functionRef:
        name: function-kcl
      input:
        apiVersion: krm.kcl.dev/v1alpha1
        kind: KCLInput
        spec:
          source: |
            # ... main.k contents inline, or reference an OCI module ...
    - step: auto-ready
      functionRef:
        name: function-auto-ready

function-auto-ready is the small but essential companion: it marks the XR Ready once its composed resources are ready, which the legacy engine did implicitly but the pipeline does not.

6. Claims, namespaces, and multi-tenancy

XRs are cluster-scoped, which is wrong for tenant self-service. The Claim is the namespaced, consumer-facing front door you got for free by setting claimNames on the XRD. An application team applies this into their namespace:

# claim.yaml  (lives in the team's namespace)
apiVersion: platform.acme.io/v1alpha1
kind: PostgreSQLInstance
metadata:
  name: orders-db
  namespace: team-orders
spec:
  parameters:
    size: medium
    storageGB: 100
  writeConnectionSecretToRef:
    name: orders-db-conn

The claim creates a backing cluster-scoped XR, the XR renders MRs, and the resulting connection secret lands as orders-db-conn in team-orders. Tenant isolation comes from standard Kubernetes primitives layered on top:

• RBAC scoped per namespace: a team can create PostgreSQLInstance claims in team-orders and nowhere else. They cannot touch XRs or MRs directly.
• Constraint policy with Kyverno or Gatekeeper on the claim CRD: enforce mandatory cost-center labels, cap storageGB per environment, or block size: large outside production.
• ResourceQuota is not enough on its own because cloud spend is not a cluster resource; the policy layer on claims is where you enforce the real guardrails.

This is the payoff. The team’s mental model is “I own a PostgreSQLInstance,” and kubectl get postgresqlinstance -n team-orders is a true inventory of what they have.

7. Package and version Configurations and Providers as xpkg images

Loose YAML in a repo is a prototype, not a platform. Bundle your XRDs, Compositions, and Functions into a Configuration package, an OCI image (.xpkg) you version, sign, and roll out like any other artifact. Define the package metadata with dependencies it needs at install time:

# crossplane.yaml
apiVersion: meta.pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: platform-postgres
spec:
  crossplane:
    version: ">=v1.18.0"
  dependsOn:
    - provider: xpkg.upbound.io/upbound/provider-aws-rds
      version: ">=v1.21.0"
    - function: xpkg.upbound.io/crossplane-contrib/function-patch-and-transform
      version: ">=v0.7.0"

Build and push with the crossplane CLI. The build packages every Crossplane resource in the directory into a single image:

# Build the xpkg from the directory holding crossplane.yaml + XRDs + Compositions
crossplane xpkg build \
  --package-root=. \
  --package-file=platform-postgres.xpkg

# Push to your registry, tagged like any OCI artifact
crossplane xpkg push \
  --package-files=platform-postgres.xpkg \
  registry.acme.io/platform/postgres:v1.2.0

Now a Configuration object is all a downstream cluster needs; Crossplane resolves and pulls the declared provider and function dependencies automatically:

apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: platform-postgres
spec:
  package: registry.acme.io/platform/postgres:v1.2.0

Versioning discipline mirrors any API: a backward-compatible Composition change is a patch or minor bump; an XRD schema change that removes or renames a field is a new XRD version (v1alpha1 → v1beta1) with both versions served during migration. Never reuse a tag.

8. GitOps delivery, upgrade, and rollback

The control plane’s desired state belongs in Git, reconciled by Argo CD or Flux. The cluster holds two distinct GitOps layers, and keeping them separate is what makes upgrades safe:

• Layer 1 (platform): Provider, Function, and Configuration objects, plus ProviderConfig and DeploymentRuntimeConfig. This is the API itself. Owned by the platform team, gated by review.
• Layer 2 (tenants): Claims. Owned by application teams in their own repos or directories, synced into their namespaces.

A package upgrade is a one-line change to a Configuration’s tag in Layer 1. Crossplane installs the new package revision alongside the old, then activates it. Because packages are immutable revisions, rollback is repointing the tag in Git and letting Argo sync; Crossplane reactivates the prior ConfigurationRevision. Control activation explicitly to avoid surprise jumps:

apiVersion: pkg.crossplane.io/v1
kind: Configuration
metadata:
  name: platform-postgres
spec:
  package: registry.acme.io/platform/postgres:v1.2.0
  revisionActivationPolicy: Manual
  revisionHistoryLimit: 3

With Manual activation, a new revision installs but stays inactive until you flip it, giving you a window to validate rendered output before live XRs reconcile against the new Composition. The critical safety property: upgrading a Composition does not delete and recreate live cloud resources. The MR controllers diff the new desired state against existing infrastructure and converge in place, so a tightened instance class is an RDS modify, not a destroy. Always dry-run the new Composition against a representative XR before activation.

Verify

Walk the full path from API definition to live infrastructure and confirm each layer reconciled.

# 1. Core, providers, and functions are healthy
kubectl get providers,functions
kubectl get pkgrev   # every revision should be Healthy + Active

# 2. The API surface exists
kubectl get xrd
kubectl get crd | grep platform.acme.io

# 3. Apply a claim and watch it resolve through the layers
kubectl apply -f claim.yaml
kubectl get postgresqlinstance -n team-orders     # the claim
kubectl get xpostgresqlinstance                    # the backing XR
kubectl get instance.rds.aws.upbound.io            # the rendered MR

# 4. Trace composition rendering and any reconcile errors
kubectl describe xpostgresqlinstance <name>        # Synced/Ready conditions + events

# 5. Confirm the connection secret was published to the tenant namespace
kubectl get secret orders-db-conn -n team-orders \
  -o jsonpath='{.data.endpoint}' | base64 -d

Render a Composition locally before it ever touches the cluster. crossplane render runs the function pipeline against an example XR and prints the MRs it would produce, which is your fast feedback loop and your pre-activation safety check:

crossplane render xr.yaml composition-aws.yaml functions.yaml

A healthy system shows SYNCED=True and READY=True on the claim, the XR, and every MR, and the published secret resolves to a real RDS endpoint.

Checklist

• Crossplane core installed; provider and function packages report Healthy.
• Provider authenticates via IRSA / Pod Identity, not static keys; IAM policy scoped to tagged resources.
• XRD models intent (t-shirt sizes), constrains input with enum/minimum/maximum, and defines a stable status output contract.
• Composition runs in Pipeline mode; patch directions and connection details verified with crossplane render.
• Loops and conditionals live in a Composition Function (KCL/Go), not contorted patches; function-auto-ready is in the pipeline.
• Claims are namespaced; per-namespace RBAC and Kyverno/Gatekeeper policy enforce tenant guardrails.
• XRDs, Compositions, and Functions packaged as a versioned, signed Configuration xpkg; tags are immutable.
• Platform layer and tenant layer reconciled by separate GitOps owners; revisionActivationPolicy: Manual for controlled upgrades and tag-repoint rollback.
• Upgrade validated to modify in place, not destroy/recreate, against a representative XR.