A signature you do not verify is just metadata. Plenty of teams wire cosign sign into CI, watch the green check, and call the supply chain “secured” — while the cluster happily pulls and runs whatever digest a Deployment points at. The control that actually matters lives at the admission boundary: a gate that refuses to run an image unless it was signed by the identity you expect, carrying the provenance you require.
This article builds that gate end to end with keyless Sigstore. We’ll sign without managing a single private key, attach SLSA provenance and an SBOM as attestations, and enforce signer identity plus predicate content with the Sigstore policy controller using ClusterImagePolicy. Then we’ll cover the parts vendors gloss over: air-gapped trust roots, break-glass, staged rollout, and auditing the transparency log for signers you never authorized.
1. Keyless signing internals: Fulcio, OIDC, and Rekor
Keyless does not mean unsigned. It means ephemeral keys bound to an identity instead of a long-lived key you have to store, rotate, and eventually leak.
The flow when CI runs cosign sign:
- Cosign generates an ephemeral keypair in memory, valid for seconds.
- It obtains an OIDC identity token from the workload’s environment — GitHub Actions, GitLab, an SPIFFE provider, or a human via browser.
- It sends the token plus the ephemeral public key to Fulcio, the certificate authority. Fulcio validates the token, then issues a short-lived X.509 certificate (~10 minutes) whose SAN encodes the OIDC
subjectand whose extension records theissuer. - Cosign signs the artifact digest with the ephemeral private key.
- The signature, certificate, and a timestamp are recorded in Rekor, the append-only transparency log. Rekor returns an inclusion proof.
- The ephemeral private key is discarded. There is nothing left to steal.
The identity is the trust anchor. A verifier later asks: “was this signed by certificate-identity X, issued by OIDC issuer Y, and is that event in Rekor?” The whole model collapses to two strings — subject and issuer — plus a transparency-log lookup. Get those two strings wrong in your policy and you have decorative cryptography.
The public-good Sigstore instance (
fulcio.sigstore.dev,rekor.sigstore.dev) is rate-limited and best-effort. Treat it as fine for OSS and experimentation, and plan to self-host for anything you’d page someone over. We cover that in section 6.
2. Signing keylessly in CI with cosign and OIDC
The non-negotiable prerequisite is an OIDC token the CI environment can mint without secrets. On GitHub Actions that’s the id-token: write permission; cosign auto-detects the ambient token, so no COSIGN_EXPERIMENTAL flag and no key material are needed.
# .github/workflows/build-sign.yml
name: build-sign
on:
push:
tags: ["v*"]
permissions:
contents: read
packages: write
id-token: write # required: mints the OIDC token Fulcio trusts
jobs:
build:
runs-on: ubuntu-latest
env:
IMAGE: ghcr.io/${{ github.repository }}
steps:
- uses: actions/checkout@v4
- uses: sigstore/cosign-installer@v3 # pins a verified cosign binary
- name: Log in to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push by digest
id: build
run: |
docker build -t "$IMAGE:${GITHUB_REF_NAME}" .
docker push "$IMAGE:${GITHUB_REF_NAME}"
DIGEST=$(docker inspect --format='{{index .RepoDigests 0}}' "$IMAGE:${GITHUB_REF_NAME}")
echo "digest=${DIGEST}" >> "$GITHUB_OUTPUT"
- name: Keyless sign (by digest, never by tag)
run: cosign sign --yes "${{ steps.build.outputs.digest }}"
Two habits that separate this from a demo:
- Sign the digest, not the tag. Tags are mutable;
cosign sign ghcr.io/app:v1signs whateverv1resolves to now. Always resolve toname@sha256:...first and sign that immutable reference. - Pin the installer and the build.
cosign-installer@v3fetches a known-good binary; if your own toolchain is compromised, your signatures are worthless no matter how good the policy is.
Blobs (Helm charts, Terraform plans, release tarballs, SBOM files) sign the same way, and the artifact moves separately from its signature:
cosign sign-blob --yes \
--bundle artifact.bundle \
release.tar.gz
# verification consumes the bundle, which carries the cert + Rekor entry
3. Attaching SLSA provenance and SBOM attestations
A signature proves who. An attestation proves what — a signed, typed statement (an in-toto predicate) bound to the same digest. Two predicates carry their weight in audits: SLSA provenance (how the artifact was built) and an SBOM (what’s inside it).
The cleanest way to get trustworthy provenance is to not hand-roll it. The SLSA GitHub generator runs in an isolated reusable workflow and emits a provenance attestation at SLSA Build Level 3 — the builder identity is the workflow itself, which is exactly what you’ll pin in policy later.
provenance:
needs: build
permissions:
actions: read
id-token: write
packages: write
uses: slsa-framework/slsa-github-generator/.github/workflows/generator_container_slsa3.yml@v2.0.0
with:
image: ghcr.io/${{ github.repository }}
digest: ${{ needs.build.outputs.digest }}
secrets:
registry-username: ${{ github.actor }}
registry-password: ${{ secrets.GITHUB_TOKEN }}
For the SBOM, generate it with Syft and attach it as a keyless attestation under the standard predicate type:
# generate a CycloneDX SBOM for the exact digest
syft "ghcr.io/myorg/app@sha256:abc123..." -o cyclonedx-json > sbom.cdx.json
# attach it as a signed, keyless attestation
cosign attest --yes \
--predicate sbom.cdx.json \
--type cyclonedx \
"ghcr.io/myorg/app@sha256:abc123..."
Each cosign attest produces its own DSSE envelope, signed via Fulcio and logged in Rekor — independently verifiable from the image signature. You now have three claims on one digest: a signature, provenance, and an SBOM. The cluster will demand all three.
4. Deploying the policy controller and writing ClusterImagePolicy
The Sigstore policy controller is a validating admission webhook. It intercepts pod-creating resources, resolves every image to a digest, and checks each against the ClusterImagePolicy objects that match it.
helm repo add sigstore https://sigstore.github.io/helm-charts
helm repo update
helm install policy-controller -n cosign-system --create-namespace \
sigstore/policy-controller
Enforcement is opt-in per namespace — a deliberate design that lets you onboard gradually. The controller only evaluates namespaces carrying its include label:
kubectl label namespace production policy.sigstore.dev/include=true
Now the core policy. This one requires that any ghcr.io/myorg/* image was signed keylessly by a specific GitHub Actions workflow, authenticated by GitHub’s OIDC issuer:
apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
name: require-keyless-signature
spec:
images:
- glob: "ghcr.io/myorg/**"
authorities:
- name: github-actions-signer
keyless:
url: https://fulcio.sigstore.dev
identities:
- issuer: "https://token.actions.githubusercontent.com"
subject: "https://github.com/myorg/app/.github/workflows/build-sign.yml@refs/tags/v1.0.0"
ctlog:
url: https://rekor.sigstore.dev
mode: enforce
Two evaluation rules govern everything and are worth committing to memory:
- Across policies that match an image: results are ANDed. Every matching
ClusterImagePolicymust pass. - Within a single policy:
authoritiesare ORed. Any one satisfied authority validates that policy. Use this for key rotation — list the old and new signer side by side during a cutover.
Pinning subject to an exact tag is brittle across releases. Use subjectRegExp to accept any tag from the trusted workflow while still rejecting every other identity:
identities:
- issuer: "https://token.actions.githubusercontent.com"
subjectRegExp: "^https://github\\.com/myorg/app/\\.github/workflows/build-sign\\.yml@refs/tags/.*$"
5. Verifying signer identity, issuer, and attestation predicates
Requiring a signature is table stakes. The real control is requiring the right provenance content. The policy controller verifies the attestation signature and then runs a CUE or Rego policy against the decoded predicate, so you can assert facts inside the SLSA statement — not merely that one exists.
This policy demands a SLSA provenance attestation whose builder is your trusted SLSA workflow, signed keylessly:
apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
name: require-slsa-provenance
spec:
images:
- glob: "ghcr.io/myorg/**"
authorities:
- name: slsa-attestation
keyless:
url: https://fulcio.sigstore.dev
identities:
- issuer: "https://token.actions.githubusercontent.com"
subjectRegExp: "^https://github\\.com/slsa-framework/slsa-github-generator/.*$"
attestations:
- name: must-be-slsa-built
predicateType: https://slsa.dev/provenance/v1
policy:
type: cue
data: |
predicate: {
buildDefinition: {
buildType: =~"https://github.com/slsa-framework/slsa-github-generator.*"
}
}
mode: enforce
The predicateType selects which attestation to evaluate; the CUE block then constrains its body — here, that the buildType came from the SLSA generator. You can extend the same pattern to require a CycloneDX SBOM attestation (predicateType: https://cyclonedx.org/bom) and assert, for example, that a specific component or license is or is not present. This is where “we have an SBOM” becomes “we enforce what the SBOM says.”
6. Air-gapped and self-hosted Sigstore
The public instances depend on a TUF (The Update Framework) root delivered over the internet — a non-starter for air-gapped clusters or anyone who refuses to make signing availability someone else’s SLA. Self-hosting means running your own Fulcio, Rekor, and CTLog, and distributing your own trust root.
Sign in CI against the internal instances by overriding the URLs:
cosign sign --yes \
--fulcio-url https://fulcio.internal.example.com \
--rekor-url https://rekor.internal.example.com \
"registry.internal.example.com/app@sha256:abc123..."
On the cluster side, the controller learns to trust your CA and log through the TrustRoot CRD. For a connected mirror you supply the initial root.json and a mirror URL; for a truly air-gapped cluster you embed the entire TUF repository inline as a base64, gzipped tarball via repository.mirrorFS, so the controller never makes an outbound call:
apiVersion: policy.sigstore.dev/v1alpha1
kind: TrustRoot
metadata:
name: internal-sigstore
spec:
repository:
root: | # base64-encoded initial root.json
<BASE64_ROOT_JSON>
mirrorFS: | # base64 of the gzipped, tarred TUF repository (air-gap)
<BASE64_REPOSITORY_TGZ>
Reference the TrustRoot from a policy’s keyless authority with trustRootRef, and the controller validates internal signatures with zero internet egress. Self-hosting buys you control and an availability story you own — at the cost of running a CA and a transparency log, which is real operational weight. Size that before committing.
7. Break-glass, exemptions, and staged enforcement
Ship mode: enforce cluster-wide on day one and you will be the reason a Sev1 mitigation can’t deploy. Roll out in stages.
Warn first. mode: warn runs the full evaluation and surfaces failures as admission warnings, but admits the pod anyway. Run here for a release cycle and watch which workloads would have been blocked:
spec:
mode: warn # logs + warns, never blocks — your dry run
Decide the no-match behavior deliberately. By default, an image matched by no policy in an enforced namespace is rejected (fail-closed) — correct for production. During onboarding, relax it via the controller config so unmatched images warn instead:
# policy-controller config: no-match-policy = warn | allow | deny
data:
no-match-policy: "warn"
Exempt what genuinely can’t be signed — third-party sidecars, vendored base images — narrowly, with a static authority that passes without verification. Scope the glob tightly; a broad static-pass policy is a backdoor:
apiVersion: policy.sigstore.dev/v1beta1
kind: ClusterImagePolicy
metadata:
name: allow-known-sidecars
spec:
images:
- glob: "registry.k8s.io/sig-storage/csi-node-driver-registrar*"
authorities:
- name: trusted-third-party
static:
action: pass
Break-glass is the label, not editing policies under pressure. Keep a pre-approved break-glass namespace whose include label is removed, with tight RBAC and an alert that fires the moment anything lands there. Pulling the label on a normal namespace to unblock an incident is the emergency lever — fast, reversible, and auditable — far safer than hand-editing a ClusterImagePolicy at 3 a.m.
8. Auditing the transparency log
Enforcement keeps bad images out today. The transparency log answers a different question: who has ever signed as us? Because every keyless signature lands in Rekor, you can hunt for signing identities you never authorized — a leaked OIDC path, a rogue workflow, a typo’d subject that still validates somewhere.
Query Rekor for every entry tied to an identity and confirm the issuer is exactly what you expect:
# every Rekor entry for a given signer identity
rekor-cli search --email "https://github.com/myorg/app/.github/workflows/build-sign.yml@refs/tags/v1.0.0"
# inspect a specific log index — confirm subject AND issuer
rekor-cli get --log-index 184392011 --format json \
| jq '.body.HashedRekordObj // .body'
For a continuous control, periodically pull entries for your image repos and diff observed (subject, issuer) pairs against an allowlist. Anything off-list is an alert — it means something obtained an OIDC token for your identity and signed with it. That is the earliest possible signal of a build-system compromise, and the transparency log is the only place you’ll see it before the artifact reaches a cluster.
Enterprise scenario
A payments platform team ran the policy controller in enforce across ~40 production namespaces, pinning the signer subject to each release tag. It worked until a Friday incident required an out-of-band hotfix built from a branch, not a tag. The exact-tag subject match rejected the image; admission blocked the rollout; the mitigation stalled while a sleepy on-call tried to edit ClusterImagePolicy objects under pressure — exactly the failure mode you build this to avoid.
The constraint: keep strict identity enforcement for normal releases, but never let the gate itself become the outage.
Two changes fixed it. First, they loosened the production policy from an exact-tag subject to a subjectRegExp that trusts the build-sign workflow on any ref from their org, so legitimate hotfix builds still verify:
identities:
- issuer: "https://token.actions.githubusercontent.com"
subjectRegExp: "^https://github\\.com/payments/.+/\\.github/workflows/build-sign\\.yml@refs/(heads|tags)/.+$"
Second, they stopped treating policy edits as the emergency procedure. They provisioned a locked-down break-glass namespace — no include label, RBAC restricted to two SREs, and a PagerDuty alert plus a Rekor audit job wired to any pod created there. Break-glass became a labelled, audited deploy target reachable in seconds, and the enforcement policies themselves stayed immutable during incidents. Mean time to mitigate for signing-related blocks dropped from “however long it takes to safely hand-edit a webhook policy” to under two minutes.
Verify
Confirm the gate actually rejects and admits the right things before you trust it.
# 1. Verify the signature locally with the exact identity you'll enforce
cosign verify \
--certificate-identity-regexp "^https://github.com/myorg/app/.*$" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
"ghcr.io/myorg/app@sha256:abc123..." | jq '.[0].optional'
# 2. Verify the SLSA provenance attestation
cosign verify-attestation \
--type slsaprovenance \
--certificate-identity-regexp "^https://github.com/slsa-framework/.*$" \
--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
"ghcr.io/myorg/app@sha256:abc123..."
# 3. Negative test: an unsigned image MUST be denied in an enforced namespace
kubectl run rogue --image=nginx:latest -n production
# expected: admission webhook "policy.sigstore.dev" denied the request:
# no matching signatures / failed policy: require-keyless-signature
# 4. Positive test: your signed image admits cleanly
kubectl run app --image="ghcr.io/myorg/app@sha256:abc123..." -n production
If step 3 admits the pod, the namespace label is missing or no-match-policy is allow — your gate is open. Fix that before anything else.