Cloud-Native CI with Tekton Pipelines and Signed Provenance via Tekton Chains

Most CI systems are a black box bolted to the side of your cluster: a YAML dialect on someone else’s controller, scaling logic you cannot see, provenance that is at best a log line. Tekton inverts that. Every build step is a pod, every pipeline is a custom resource, and the same RBAC, admission, and observability you run for workloads applies to your CI. Once builds are Kubernetes objects, emitting tamper-evident SLSA provenance stops being a pipeline stage and becomes a controller that watches PipelineRun completions. This article builds reusable pipelines from the CRDs up, then wires Tekton Chains so every artifact is signed and every build produces an in-toto attestation without a single extra task.

1. The Tekton CRD model: Task, Pipeline, PipelineRun, results

Tekton is four core nouns. A Task is an ordered list of steps, each a container. A Pipeline arranges Tasks into a DAG with explicit ordering and data flow. A PipelineRun (or TaskRun) is the execution — it binds parameters, workspaces, and a service account, and it is the object Chains later signs. Task and Pipeline are reusable templates with zero runtime state; the Run objects hold all instance data.

Install the core component and confirm the API is serving:

kubectl apply -f https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

# Wait for the controller and webhook to be Ready
kubectl wait --for=condition=Ready pods --all -n tekton-pipelines --timeout=180s
kubectl api-resources --api-group=tekton.dev

You should see tasks, pipelines, pipelineruns, and taskruns under tekton.dev/v1 — the stable API you should author against. v1beta1 still resolves via conversion but is deprecated.

The two data primitives that make Tasks composable are results and workspaces. A result is a small string a step writes to $(results.<name>.path) that downstream Tasks consume as $(tasks.<task>.results.<name>). Results are for facts — a digest, a tag, a commit SHA — and are capped at roughly 4 KB total per TaskRun when stored in the termination message. A workspace is a shared filesystem (a PVC, emptyDir, Secret, or ConfigMap) mounted across steps and Tasks for bulk data like source trees and caches.

A minimal but real Task that clones a repo and emits the resolved commit as a result:

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: git-clone-min
spec:
  params:
    - name: url
      type: string
    - name: revision
      type: string
      default: "main"
  workspaces:
    - name: source
      description: Where the repo is checked out
  results:
    - name: commit
      description: The resolved commit SHA
  steps:
    - name: clone
      image: cgr.dev/chainguard/git:latest
      script: |
        #!/usr/bin/env sh
        set -eu
        cd "$(workspaces.source.path)"
        git clone "$(params.url)" .
        git checkout "$(params.revision)"
        git rev-parse HEAD | tr -d '\n' > "$(results.commit.path)"

2. Reusable Tasks and pulling shared ones from Tekton Hub

You should not hand-write a git-clone Task in production. The community maintains hardened, parameterized Tasks on Tekton Hub. Install the tkn CLI and pull one in:

# Install the official git-clone Task (cluster-scoped, namespaced install)
tkn hub install task git-clone --version 0.9
tkn task list

For supply-chain hygiene, pin a specific catalog version rather than tracking latest, and review the Task source — a Task is arbitrary container execution with whatever service account you bind. The catalog git-clone Task exposes results you rely on downstream: commit, url, and committer-date.

The better pattern for fleet-wide reuse is Tekton Resolvers, which lets a Pipeline reference a Task by remote ref instead of vendoring YAML. The hub and git resolvers are both built in:

# Inside a Pipeline spec, reference a remote Task without copying it locally
- name: fetch-source
  taskRef:
    resolver: hub
    params:
      - name: kind
        value: task
      - name: name
        value: git-clone
      - name: version
        value: "0.9"
  workspaces:
    - name: output
      workspace: shared-data
  params:
    - name: url
      value: $(params.repo-url)

This keeps a single source of truth: bump the version in one Pipeline, not fifty copied manifests. Use the git resolver for your own internal Tasks — point it at a tag or commit in your platform repo so the resolved Task is pinned and auditable.

3. Sharing data with workspaces, volumes, and result passing

A Pipeline ties Tasks together along two axes: ordering (runAfter or implicit results dependencies) and data (workspaces and results). The pattern below clones into a shared workspace, builds an image with Kaniko, and passes the digest forward as a result so later Tasks — and Chains — can reference the exact artifact.

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: build-and-push
spec:
  params:
    - name: repo-url
      type: string
    - name: image-ref
      type: string
  workspaces:
    - name: shared-data
    - name: docker-credentials
  tasks:
    - name: fetch-source
      taskRef:
        resolver: hub
        params:
          - name: kind
            value: task
          - name: name
            value: git-clone
          - name: version
            value: "0.9"
      workspaces:
        - name: output
          workspace: shared-data
      params:
        - name: url
          value: $(params.repo-url)
    - name: build-push
      runAfter: ["fetch-source"]
      taskRef:
        resolver: hub
        params:
          - name: kind
            value: task
          - name: name
            value: kaniko
          - name: version
            value: "0.6"
      workspaces:
        - name: source
          workspace: shared-data
        - name: dockerconfig
          workspace: docker-credentials
      params:
        - name: IMAGE
          value: $(params.image-ref)
  results:
    - name: image-digest
      value: $(tasks.build-push.results.IMAGE_DIGEST)

The Pipeline-level results block re-exports a Task result so it appears on the PipelineRun status. This matters for Chains, which keys off Pipeline parameters and results to know what was built. The catalog kaniko Task already writes IMAGE_DIGEST and IMAGE_URL — that is not an accident, it is the convention Chains expects.

For workspace backing, use a volumeClaimTemplate so each PipelineRun gets its own ephemeral PVC garbage-collected with the run, rather than a shared PVC that serializes concurrent builds:

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  generateName: build-and-push-
spec:
  pipelineRef:
    name: build-and-push
  taskRunTemplate:
    serviceAccountName: tekton-builder
  params:
    - name: repo-url
      value: https://github.com/acme/widget-api
    - name: image-ref
      value: registry.acme.io/widget-api:$(context.pipelineRun.uid)
  workspaces:
    - name: shared-data
      volumeClaimTemplate:
        spec:
          accessModes: ["ReadWriteOnce"]
          resources:
            requests:
              storage: 1Gi
    - name: docker-credentials
      secret:
        secretName: registry-creds

4. Triggering pipelines from webhooks with EventListeners

A PipelineRun you apply by hand is a demo; production CI fires on git push. Tekton Triggers turns an inbound webhook into a PipelineRun via three nouns: EventListener (an HTTP sink backed by a pod and Service), TriggerBinding (extracts fields from the payload), and TriggerTemplate (the parameterized object to create). Interceptors sit in front to validate, filter, and enrich.

kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml
kubectl apply -f https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml
kubectl wait --for=condition=Ready pods --all -n tekton-pipelines --timeout=180s

The wiring below validates the GitHub HMAC signature, fires only on pushes to main, binds the repo URL and commit, and stamps out a PipelineRun:

apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerBinding
metadata:
  name: github-push-binding
spec:
  params:
    - name: repo-url
      value: $(body.repository.clone_url)
    - name: revision
      value: $(body.after)
---
apiVersion: triggers.tekton.dev/v1beta1
kind: TriggerTemplate
metadata:
  name: build-trigger-template
spec:
  params:
    - name: repo-url
    - name: revision
  resourcetemplates:
    - apiVersion: tekton.dev/v1
      kind: PipelineRun
      metadata:
        generateName: build-and-push-
      spec:
        pipelineRef:
          name: build-and-push
        taskRunTemplate:
          serviceAccountName: tekton-builder
        params:
          - name: repo-url
            value: $(tt.params.repo-url)
          - name: image-ref
            value: registry.acme.io/widget-api:$(tt.params.revision)
        workspaces:
          - name: shared-data
            volumeClaimTemplate:
              spec:
                accessModes: ["ReadWriteOnce"]
                resources:
                  requests:
                    storage: 1Gi
          - name: docker-credentials
            secret:
              secretName: registry-creds
---
apiVersion: triggers.tekton.dev/v1beta1
kind: EventListener
metadata:
  name: github-listener
spec:
  serviceAccountName: tekton-triggers-sa
  triggers:
    - name: github-push
      interceptors:
        - ref:
            name: "github"
          params:
            - name: secretRef
              value:
                secretName: github-webhook-secret
                secretKey: secretToken
            - name: eventTypes
              value: ["push"]
        - ref:
            name: "cel"
          params:
            - name: filter
              value: "body.ref == 'refs/heads/main'"
      bindings:
        - ref: github-push-binding
      template:
        ref: build-trigger-template

The github interceptor performs the HMAC check — this is your authentication boundary, so the secret must be a real random token configured identically in the GitHub webhook. The cel interceptor is where branch and path filtering belongs; pushing that logic upstream means you never spin up a pod for an irrelevant event. Expose the EventListener Service (Ingress, or a Route on OpenShift) and register that URL as the webhook target.

5. Generating SLSA provenance automatically with Tekton Chains

Here is the payoff. Tekton Chains is a controller that watches TaskRun and PipelineRun objects; on completion it observes their inputs (parameters), outputs (results), and produced image references, then generates a signed in-toto attestation describing exactly what was built from what. You do not add a task; you install a controller and configure it.

kubectl apply -f https://storage.googleapis.com/tekton-releases/chains/latest/release.yaml
kubectl wait --for=condition=Ready pods --all -n tekton-chains --timeout=180s

Chains is configured entirely through the chains-config ConfigMap in tekton-chains. The defaults are conservative; you want the SLSA provenance format and storage in the OCI registry alongside the image:

kubectl patch configmap chains-config -n tekton-chains -p '{
  "data": {
    "artifacts.taskrun.format": "slsa/v2alpha4",
    "artifacts.taskrun.storage": "oci",
    "artifacts.pipelinerun.format": "slsa/v2alpha4",
    "artifacts.pipelinerun.storage": "oci",
    "artifacts.oci.storage": "oci",
    "artifacts.oci.format": "simplesigning",
    "transparency.enabled": "true"
  }
}'

# Restart the controller to pick up config changes
kubectl rollout restart deployment tekton-chains-controller -n tekton-chains

What each key does:

For Chains to recognize what a TaskRun built, the run must expose results it can map to artifacts. With the catalog kaniko Task that means IMAGE_URL and IMAGE_DIGEST; Chains reads those, fetches the image, and signs it. This is why the result naming in step 3 was load-bearing — Chains is convention-driven, and getting the names right is the difference between a signed artifact and a silent no-op.

6. Signing artifacts and attestations with cosign and KMS

Chains needs a key to sign with. For a lab, generate a cosign keypair stored as a Kubernetes Secret in the Chains namespace — Chains looks for a Secret named signing-secrets:

# cosign writes directly to the secret Chains expects
cosign generate-key-pair k8s://tekton-chains/signing-secrets

That populates cosign.key, cosign.pub, and cosign.password inside signing-secrets. A static key on the cluster proves the flow, but in production you do not want a long-lived private key in etcd. Point Chains at a cloud KMS instead, so the private key never leaves the HSM:

kubectl patch configmap chains-config -n tekton-chains -p '{
  "data": {
    "signers.kms.kmsref": "gcpkms://projects/acme-prod/locations/us-central1/keyRings/tekton/cryptoKeys/chains-signer/versions/1"
  }
}'
kubectl rollout restart deployment tekton-chains-controller -n tekton-chains

Chains supports the cosign KMS reference scheme across providers — gcpkms://, awskms://, azurekms://, and hashivault://. The controller’s workload identity (IRSA on EKS, a workload-identity binding on GKE) needs sign and get-public-key permission on that key and nothing more. The fully keyless path also works: set signers.x509.fulcio.enabled and Chains requests a short-lived certificate from Fulcio bound to the controller’s OIDC identity — no key material at all, at the cost of a hard dependency on a Fulcio instance.

7. Securing PipelineRuns: service accounts, pod security, limits

CI is the highest-value target in your cluster — it can push to your registry and often holds cloud credentials. Treat it that way.

Least-privilege service accounts. The builder SA needs registry push and nothing else; the triggers SA needs to create PipelineRuns and read its triggers resources. Keep them split:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: tekton-builder
  namespace: ci
secrets:
  - name: registry-creds
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: tekton-triggers-createonly
  namespace: ci
rules:
  - apiGroups: ["tekton.dev"]
    resources: ["pipelineruns", "taskruns"]
    verbs: ["create"]
  - apiGroups: ["triggers.tekton.dev"]
    resources: ["eventlisteners", "triggerbindings", "triggertemplates", "triggers", "clusterinterceptors", "interceptors"]
    verbs: ["get", "list", "watch"]

Pod Security and step hardening. Label the CI namespace for the restricted Pod Security Standard. Kaniko historically needed a relaxed profile because it manipulates the filesystem; prefer rootless build tooling (Kaniko rootless mode, or Buildah running unprivileged) so the namespace can stay restricted:

apiVersion: v1
kind: Namespace
metadata:
  name: ci
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/enforce-version: latest

Resource limits. Every step is a container; without limits one runaway build starves the node. Set computeResources so the scheduler can bin-pack and the kubelet can evict fairly:

steps:
  - name: build
    image: gcr.io/kaniko-project/executor:latest
    computeResources:
      requests:
        cpu: "500m"
        memory: 1Gi
      limits:
        cpu: "2"
        memory: 4Gi

A final non-obvious control: enable enforce-nonfalsifiable: true in Chains config. It hashes the TaskRun spec into the provenance, so a run mutated mid-flight (for example by a compromised admission webhook) cannot produce a clean attestation.

8. Sharding executions, pruning runs, and dashboard observability

At fleet scale, three operational facts bite. First, completed PipelineRun and TaskRun objects accumulate in etcd and never leave on their own — you must prune them:

# Imperative cleanup: keep the 50 most recent runs in the ci namespace,
# delete the rest (wrap in a CronJob for unattended pruning)
tkn pipelinerun delete --keep 50 --all -n ci

For declarative pruning under the Tekton Operator, the TektonConfig CR exposes a pruner block (schedule plus keep/keep-since and which resources to prune) — the supported way to bound history without a hand-rolled CronJob.

Second, a single controller pod is a throughput ceiling. Tekton supports HA via leader election with sharded buckets — increase buckets in config-leader-election and scale the controller Deployment so reconciliation is partitioned across replicas:

kubectl patch configmap config-leader-election -n tekton-pipelines \
  -p '{"data":{"buckets":"3"}}'
kubectl scale deployment tekton-pipelines-controller -n tekton-pipelines --replicas=3

Third, observability. Install the Tekton Dashboard for a read-only view of runs, logs, and the DAG, and scrape the controller’s Prometheus metrics for tekton_pipelines_controller_pipelinerun_duration_seconds and reconcile latency:

kubectl apply -f https://storage.googleapis.com/tekton-releases/dashboard/latest/release.yaml
kubectl port-forward -n tekton-pipelines svc/tekton-dashboard 9097:9097

Verify

Run an end-to-end build and prove the artifact is signed and carries provenance.

# 1. Fire a build
kubectl create -f pipelinerun.yaml -n ci
tkn pipelinerun logs --last -f -n ci

# 2. Confirm Chains signed it — the run gets annotated when signing succeeds
kubectl get pipelinerun --sort-by=.metadata.creationTimestamp -n ci -o jsonpath \
  '{range .items[-1:]}{.metadata.annotations.chains\.tekton\.dev/signed}{"\n"}{end}'
# Expect: true

# 3. Verify the image signature with the public key
cosign verify --key k8s://tekton-chains/signing-secrets \
  registry.acme.io/widget-api:<tag>

# 4. Pull and inspect the SLSA provenance attestation
cosign verify-attestation --key k8s://tekton-chains/signing-secrets \
  --type slsaprovenance \
  registry.acme.io/widget-api:<tag> | jq -r '.payload' | base64 -d | jq .

Step 2 returning true is the controller’s receipt that signing completed. Step 4 should print an in-toto statement whose predicate lists the builder ID, the materials (your git URL and resolved commit), and the invocation parameters. With transparency.enabled on, cosign verify-attestation also confirms the Rekor entry exists.

If the signed annotation never appears, the usual cause is missing or misnamed results: Chains only signs artifacts it can identify, so a Task that does not emit IMAGE_URL/IMAGE_DIGEST produces an unsigned run with no error. Check the controller logs in tekton-chains for no signable targets found.

Enterprise scenario

A platform team on a regulated payments product needed every production container to carry SLSA Build L2 provenance, but auditors raised a hard objection to the first design: cosign signing keys lived as Kubernetes Secrets, so any cluster-admin (and the etcd backup process) could exfiltrate the private key and forge provenance for a malicious image. The control was theater if the key was reachable.

They re-architected signing to be fully keyless: stood up Fulcio and Rekor instances, bound the Chains controller to a dedicated workload identity with no other permissions, and switched Chains to request short-lived Fulcio certificates instead of a static key — so there was no private key to steal, and every signature anchored to the controller’s OIDC identity plus a transparency-log entry.

kubectl patch configmap chains-config -n tekton-chains -p '{
  "data": {
    "signers.x509.fulcio.enabled": "true",
    "signers.x509.fulcio.address": "https://fulcio.internal.acme.io",
    "signers.x509.fulcio.issuer": "https://oidc.acme.io",
    "signers.x509.fulcio.provider": "spiffe",
    "transparency.enabled": "true",
    "transparency.url": "https://rekor.internal.acme.io"
  }
}'

The audit win was decisive: with no key material anywhere, “who signed this build” was answerable from the certificate identity and “was it tampered with after signing” from Rekor — without trusting a single long-lived secret. The trade-off was operating Fulcio and Rekor as tier-1 services, since a Fulcio outage now blocked production builds; they mitigated that with a regional active-passive deployment and an alert on certificate-issuance latency.

Checklist

• Tekton Pipelines installed and tekton.dev/v1 API confirmed serving
• Shared Tasks pulled from Tekton Hub with pinned versions (not latest) and reviewed
• Remote Tasks referenced via the hub/git resolver instead of vendored YAML
• Workspaces use volumeClaimTemplate for per-run isolation; results pass digests forward
• EventListener fronted by github (HMAC) and cel (branch/path) interceptors
• Tekton Chains installed; chains-config set to SLSA format with OCI storage
• Signing backed by KMS or keyless Fulcio, not a long-lived in-cluster key
• transparency.enabled: true so signatures land in Rekor
• enforce-nonfalsifiable: true to bind provenance to the immutable run spec
• Builder and triggers service accounts split to least privilege; CI namespace restricted
• Step computeResources limits set so a runaway build cannot starve the node
• PipelineRun pruning configured; controller sharded with buckets for HA
• Verified end to end: cosign verify and cosign verify-attestation both pass