Multi-Architecture Container Builds with docker buildx bake: Remote Cache, Provenance, and Registry-Native Pipelines

In a nutshell

Imagine you run a small print shop. Today, for every poster a customer orders, you walk to the machine, dial in the paper size, the ink, the finish, and press start — by hand, one poster at a time. That is docker buildx build: one image, one long line of options, repeated. docker buildx bake is the order form. You write down every poster you ever print — sizes, inks, finishes — once, in a single file, and then you just say “print the catalogue.” The machine reads the form, runs all the jobs at once, and reuses shared setup between them. That file is docker-bake.hcl, each poster is a target, and “print the catalogue” is docker buildx bake.

The second idea is multi-architecture. Computers come in two common CPU families: amd64 (most laptops, Intel/AMD servers) and arm64 (Apple Silicon Macs, AWS Graviton, Raspberry Pi, most phones). A program compiled for one will not run on the other — the CPU literally does not understand the instructions. A multi-arch image solves this by packing both builds under a single name. When someone runs docker pull myapp, Docker quietly hands them the version that matches their machine. One tag, every CPU. The thing that makes that possible is a small index file called a manifest list — a table of contents that says “for amd64 use this image, for arm64 use that one.”

Put together: bake lets you describe many image builds declaratively, and multi-arch lets each of those images run everywhere. This lesson shows you how to do both in a real CI pipeline — fast, cached, and with a signed record of exactly how each image was built.

Prerequisites and what you’ll be able to do

Level: Advanced · Time: ~35 min · you should be comfortable with a terminal and a Dockerfile.

Know this first:

• How a Dockerfile becomes an image and how layers cache — Containers & Docker basics.
• Multi-stage builds and BuildKit cache mounts — Multi-stage Dockerfiles & BuildKit cache.
• The idea of a container registry (a place you push and pull images) and basic CI concepts (a pipeline that runs on git push).

After this lesson you will be able to:

• Write a docker-bake.hcl with shared base targets, groups, variables, and matrix expansion instead of a shell loop.
• Choose the right buildx builder driver (docker, docker-container, kubernetes, remote) for the job.
• Build a true multi-arch image for linux/amd64 and linux/arm64 and understand the manifest list it produces.
• Set up registry-native remote cache that stays warm across CI runs, and debug it when it misses.
• Split each architecture onto a native runner (no QEMU) and merge the results into one tag.
• Attach SLSA provenance and an SBOM to every image at build time.

If your container CI is a shell loop calling docker buildx build once per service with a wall of --build-arg, --cache-from, and --platform flags pasted into every job, you already know the failure modes: drift between targets, cache keys nobody can reason about, and a 30-minute pipeline because arm64 is emulated under QEMU. docker buildx bake is the fix. It is a build orchestrator that reads declarative HCL, expands matrices, fans builds out in parallel, and treats the registry as the source of truth for both cache and provenance.

This is a working pipeline, not a tour of flags. Everything assumes a recent Docker Engine with Buildx v0.12+ and BuildKit v0.13+ (docker buildx version).

The pipeline at a glance

Before the details, here is the whole journey on one page — from the declarative file to a cached, signed, multi-arch image in a registry.

The diagram traces one bake run left to right: a CI pipeline invokes docker buildx bake against a declarative docker-bake.hcl; the buildx builder expands the targets and fans them out; each architecture builds on its own native runner in parallel; the per-arch images are stitched into a single manifest-list tag with provenance and an SBOM attached; and the result is pushed to an OCI registry alongside a remote cache that the next run reads back so it starts warm. Keep this shape in mind — every numbered section below fills in one band of it.

1. Why bake beats shell loops

bake is to docker buildx build what a build system is to a pile of compiler invocations. You describe targets (one image build), group them, and let bake compute the graph. The wins are concrete:

• One source of truth. Shared base config lives in a target other targets inherits. Change the registry once.
• Parallelism for free. A group builds all its targets concurrently against a single BuildKit daemon, which deduplicates shared layers across them.
• Matrix expansion. One target definition becomes N builds across versions or variants without copy-paste.
• It is just build underneath. Every HCL attribute maps to a buildx build flag, so nothing is hidden or magic.

A bake file can be HCL, JSON, or a Compose file. Use HCL — it has variables, functions, and interpolation that Compose lacks.

The building blocks of a bake file. Everything you write in HCL is one of six constructs. Learn these names and the rest of the lesson reads easily:

The mental model: variables parameterise the file, a base target holds shared attributes, concrete targets inherits it and set what differs, a matrix multiplies a target across a dimension, and a group is the button you press. bake reads all of it, resolves the graph, and hands one plan to BuildKit.

2. Set up the builder: QEMU emulation vs native arm64

The default docker driver cannot build multi-platform images or export to a registry cache. You need the docker-container driver, which runs BuildKit in its own container.

# Create a dedicated builder backed by the docker-container driver
docker buildx create \
  --name kv-builder \
  --driver docker-container \
  --driver-opt network=host \
  --bootstrap \
  --use

docker buildx inspect kv-builder

For cross-architecture builds on a single amd64 host, register QEMU so BuildKit can emulate arm64:

# binfmt_misc handlers; --install all wires up every supported arch
docker run --privileged --rm tonistiigi/binfmt --install all

Emulation works but is slow — CPU-bound steps (compilers, npm install running native postinstalls) can run 5-10x slower under QEMU. The production-grade answer is native runners per architecture: an amd64 host builds amd64, a Graviton/Ampere host builds arm64, and you merge the results. You can even attach multiple nodes to one logical builder so a single bake invocation schedules each platform on the node that matches it:

# Append a remote arm64 node to the same builder; bake routes by platform
docker buildx create \
  --name kv-builder \
  --append \
  --node kv-arm64 \
  --platform linux/arm64 \
  ssh://ci@arm64-runner.internal

Rule of thumb: emulation is fine for interpreted-language images and final assembly. For anything that compiles, pay for native arm64 capacity — it is cheaper than the pipeline minutes you burn under QEMU.

The four buildx drivers. “Builder” and “driver” trip people up, so pin them down. A builder is a named build environment; its driver decides where and how BuildKit actually runs. There are exactly four, and only the first is unable to do multi-arch:

Why can’t the default docker driver do it? BuildKit inside the engine exports straight into the local image store, which only understands a single-platform image and has no concept of a registry cache manifest. The docker-container driver runs a full standalone BuildKit that can emit an OCI image index and push cache blobs — which is why every multi-arch recipe starts with docker buildx create.

3. Write docker-bake.hcl: shared base, inheritance, and matrix

Here is the spine of a real multi-service repo. Variables read from the environment with defaults, a _common base target carries shared config, and concrete targets inherits it.

# docker-bake.hcl
variable "REGISTRY" {
  default = "ghcr.io/kloudvin"
}

variable "TAG" {
  default = "dev"
}

# Comma-separated platform list, overridable in CI
variable "PLATFORMS" {
  default = "linux/amd64,linux/arm64"
}

# Default group: what `docker buildx bake` builds with no args
group "default" {
  targets = ["api", "worker"]
}

# Abstract base. The leading underscore is convention; it is still a
# target, just one you never build directly.
target "_common" {
  context    = "."
  platforms  = split(",", PLATFORMS)
  labels = {
    "org.opencontainers.image.source" = "https://github.com/kloudvin/platform"
  }
}

target "api" {
  inherits   = ["_common"]
  dockerfile = "api/Dockerfile"
  tags       = ["${REGISTRY}/api:${TAG}"]
  args = {
    SERVICE = "api"
  }
}

target "worker" {
  inherits   = ["_common"]
  dockerfile = "worker/Dockerfile"
  tags       = ["${REGISTRY}/worker:${TAG}"]
  args = {
    SERVICE = "worker"
  }
}

Build it:

REGISTRY=ghcr.io/kloudvin TAG=1.4.0 docker buildx bake

bake builds api and worker in parallel, each for both platforms. To build one target: docker buildx bake api.

Where values come from (precedence). A variable’s final value is resolved in a clear order, lowest to highest priority: the default in the file, then a matching environment variable, then a --set on the command line. So TAG=1.4.0 docker buildx bake api --set api.tags=ghcr.io/kloudvin/api:hotfix uses 1.4.0 for everything except the tag, which --set wins. --set also accepts globs (*.platform=... hits every target) and dotted paths into maps (api.args.SERVICE=...). This is the seam you use in CI to override a file that is otherwise committed and stable.

Inheritance chains compose. inherits is a list and it is transitive: a target can inherit _common, and a _common-prod can itself inherit _common and tighten a few attributes, and a service target inherits _common-prod. Attributes set later in the chain win. This is how a monorepo keeps “the registry, the labels, the cache policy” in exactly one place while each service overrides only its dockerfile and tags.

Matrix expansion

A matrix attribute turns one target into a build per combination. The name field must be unique per cell — derive it from the matrix keys.

target "runtime" {
  inherits = ["_common"]
  name     = "runtime-${tgt.replace(".", "-")}"
  matrix = {
    tgt = ["3.11", "3.12", "3.13"]
  }
  dockerfile = "images/python/Dockerfile"
  tags       = ["${REGISTRY}/python:${tgt}"]
  args = {
    PYTHON_VERSION = tgt
  }
}

docker buildx bake runtime now produces three images, each multi-arch, all concurrent. This single block replaces a nested for loop over versions and platforms. A matrix can have more than one key — matrix = { tgt = ["3.12","3.13"], variant = ["slim","full"] } expands to the full cross-product (four cells here), and your name template must incorporate every key so each cell stays unique.

4. Registry-native remote cache

Inline cache (type=inline, the old BUILDKIT_INLINE_CACHE=1) only stores cache metadata for the final stage, so multi-stage builds miss on intermediate layers. For real CI cache, use type=registry with mode=max, which exports a cache manifest for every layer of every stage to a dedicated tag.

Add cache-to and cache-from to the base target so all targets share the policy:

target "_common" {
  context   = "."
  platforms = split(",", PLATFORMS)

  cache-from = [
    "type=registry,ref=${REGISTRY}/buildcache:api"
  ]
  cache-to = [
    "type=registry,ref=${REGISTRY}/buildcache:api,mode=max,compression=zstd"
  ]
}

Two things that bite people:

• mode=max is non-negotiable for layered builds. The default mode=min only exports the layers in the final image; your build and deps stages get no cache and rebuild every time.
• Cache is per-architecture. A registry cache ref holds a manifest list; BuildKit picks the matching arch entry. You do not need a separate ref per platform, but the cache only helps a platform if that platform was previously exported to it.

compression=zstd shrinks the cache blobs and speeds up restore. If you push images to ECR or another registry that historically lagged on OCI image-index support, also set oci-mediatypes=true and confirm the registry accepts it.

5. Split platforms across native runners, then merge

The fastest pipeline builds each architecture on a native runner and pushes them as separate per-arch tags, then stitches them into one multi-arch manifest with imagetools create. No QEMU, full parallelism across machines.

Per-arch build (one job per runner). Note --set overrides any HCL attribute from the CLI:

# On the amd64 runner
docker buildx bake api \
  --set "*.platform=linux/amd64" \
  --set "api.tags=${REGISTRY}/api:${TAG}-amd64" \
  --set "*.output=type=registry"

# On the arm64 runner
docker buildx bake api \
  --set "*.platform=linux/arm64" \
  --set "api.tags=${REGISTRY}/api:${TAG}-arm64" \
  --set "*.output=type=registry"

The * glob applies to every target; api.tags targets one. Both jobs push by digest.

Merge into a single tag in a final job. imagetools create builds a manifest list referencing the existing per-arch images without re-pulling or rebuilding — it is a registry-side operation:

docker buildx imagetools create \
  --tag ${REGISTRY}/api:${TAG} \
  ${REGISTRY}/api:${TAG}-amd64 \
  ${REGISTRY}/api:${TAG}-arm64

Inspect the result to confirm both platforms are present under one tag:

docker buildx imagetools inspect ${REGISTRY}/api:${TAG}

6. Provenance and SBOM attestations during bake

BuildKit can emit in-toto attestations — SLSA provenance and an SBOM — and push them as referrer manifests alongside the image, all during the build. Add attest entries to the target:

target "api" {
  inherits   = ["_common"]
  dockerfile = "api/Dockerfile"
  tags       = ["${REGISTRY}/api:${TAG}"]
  output     = ["type=registry"]

  attest = [
    "type=provenance,mode=max",
    "type=sbom"
  ]
}

mode=max provenance records the full build: source, materials, the Dockerfile, and build args (so do not bake secrets into args). The equivalent on the CLI is --provenance=mode=max --sbom=true, and you can force it across all targets with --set "*.attest=type=provenance,mode=max".

Attestations only attach when the export target understands the OCI referrers API — that means type=registry (or type=image,push=true). They are dropped silently for type=docker loads into the local engine. Verify after pushing:

# Lists the image plus its attestation manifests
docker buildx imagetools inspect ${REGISTRY}/api:${TAG} --format '{{json .Provenance}}'

This gives you SLSA provenance at build time; sign the resulting digest with Cosign in a later step to complete the chain.

7. Wire bake into CI with a sane cache key strategy

The pattern below uses GitHub Actions with a matrix over architecture, native runners, and a separate merge job. The key decisions: a stable cache ref so warm caches survive across branches, and concurrency to cancel superseded runs.

name: build
on:
  push:
    branches: [main]
    tags: ["v*"]

concurrency:
  group: build-${{ github.ref }}
  cancel-in-progress: true

permissions:
  contents: read
  packages: write
  id-token: write   # required for keyless signing later

jobs:
  build:
    strategy:
      fail-fast: false
      matrix:
        include:
          - platform: linux/amd64
            runner: ubuntu-24.04
            suffix: amd64
          - platform: linux/arm64
            runner: ubuntu-24.04-arm   # native arm64, no QEMU
            suffix: arm64
    runs-on: ${{ matrix.runner }}
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
        with:
          driver-opts: network=host
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Bake (per-arch)
        run: |
          docker buildx bake api \
            --set "*.platform=${{ matrix.platform }}" \
            --set "api.tags=ghcr.io/kloudvin/api:${{ github.sha }}-${{ matrix.suffix }}" \
            --set "*.cache-from=type=registry,ref=ghcr.io/kloudvin/buildcache:api-${{ matrix.suffix }}" \
            --set "*.cache-to=type=registry,ref=ghcr.io/kloudvin/buildcache:api-${{ matrix.suffix }},mode=max" \
            --set "*.output=type=registry"

  merge:
    needs: build
    runs-on: ubuntu-24.04
    steps:
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}
      - name: Merge per-arch into multi-arch tag
        run: |
          docker buildx imagetools create \
            --tag ghcr.io/kloudvin/api:${{ github.sha }} \
            ghcr.io/kloudvin/api:${{ github.sha }}-amd64 \
            ghcr.io/kloudvin/api:${{ github.sha }}-arm64

A per-architecture cache ref (buildcache:api-amd64 vs -arm64) keeps the two runners from clobbering each other’s manifest. Keying the ref by service and arch rather than by commit means the cache stays warm across pushes — BuildKit reuses unchanged layers and only rebuilds what the diff touched.

The GitLab shape is identical in spirit: a parallel:matrix over ARCH, docker buildx bake with the same --set overrides, and a dependent merge stage running imagetools create. Use a project-scoped deploy token for docker login against the GitLab registry.

8. Debug cache misses

When a build that should be cached rebuilds from scratch, work the problem in this order.

First, see the resolved plan. --print evaluates the HCL and prints the final JSON without building. This catches the most common bug: a --set or variable that changed a value participating in the cache key.

docker buildx bake api --print

Compare cache keys across runs. Cache misses almost always trace to an input that changed: a build arg, a base image digest that floated, or a COPY whose context shifted. Run with BuildKit debug output to watch which steps resolve as CACHED versus re-executed:

BUILDKIT_PROGRESS=plain docker buildx bake api 2>&1 | grep -E "CACHED|exporting"

Inspect what the registry cache actually contains. If cache-from finds nothing, confirm the cache tag exists and holds your architecture:

docker buildx imagetools inspect ghcr.io/kloudvin/buildcache:api-amd64

Use the build history. Buildx records recent builds. List them and open one to see the full step-by-step trace, including cache decisions, long after the job finished:

docker buildx history ls
docker buildx history inspect <build-id>

Avoid BUILDKIT_INLINE_CACHE=1 as your primary mechanism — it is mode=min semantics and only carries final-stage cache. Keep registry cache as the source of truth and reserve inline cache for the narrow case where you want consumers of a published image to get some cache without a separate cache ref.

Going deeper

Everything above gets you a fast, correct pipeline. This section is for when you own the platform and need to reason about the internals, the trade-offs, and the failure modes at scale.

Remote cache backends compared

type=registry is the default recommendation, but BuildKit supports several cache backends and the right one depends on where your CI runs. All of them plug into the same cache-to/cache-from slots.

The gha backend deserves a note: it does not touch your registry at all, which keeps the registry clean, but it inherits GitHub’s cache quotas and eviction (roughly a 10 GB per-repo budget, least-recently-used eviction). For big monorepos that blows out fast, and type=registry on a dedicated cache repo scales better. local cache is the one that surprises people — it lives on the runner’s disk, so it is worthless the moment your CI runner is ephemeral, which hosted runners always are.

The kubernetes driver for shared CI capacity

When many pipelines build all day, per-runner docker-container builders waste warm-up time and cannot share cache in memory. The kubernetes driver runs BuildKit as pods you can autoscale, and it can pin architectures to node pools so a multi-arch build lands natively on both — no QEMU anywhere.

# amd64 build pods, pinned to amd64 nodes
docker buildx create \
  --name k8s-builder \
  --driver kubernetes \
  --driver-opt namespace=buildkit,replicas=3,"nodeselector=kubernetes.io/arch=amd64" \
  --platform linux/amd64 \
  --use

# Append arm64 build pods on Graviton/Ampere nodes to the SAME builder
docker buildx create \
  --name k8s-builder \
  --append \
  --driver kubernetes \
  --driver-opt namespace=buildkit,replicas=2,"nodeselector=kubernetes.io/arch=arm64" \
  --platform linux/arm64

Now docker buildx bake --set "*.output=type=registry" schedules the amd64 leg on amd64 pods and the arm64 leg on arm64 pods in one invocation. This is the cleanest way to give a whole engineering org native multi-arch builds without a fleet of static runners — the pods scale to zero between builds. Pair it with a shared type=registry cache so every pipeline warms the same cache.

Cross-compilation vs emulation: the performance cliff

QEMU emulation is correct but slow because it interprets every CPU instruction of the foreign architecture. For a compiler, that is the whole workload. The escape hatch, for languages that support it, is cross-compilation: run the compiler natively on the build host but tell it to emit code for the target architecture. BuildKit hands you the platform pair as automatic build args:

# Compile on the native BUILDPLATFORM, target the requested TARGETARCH.
# No QEMU: the Go toolchain itself emits arm64 from an amd64 host.
FROM --platform=$BUILDPLATFORM golang:1.23 AS build
ARG TARGETOS
ARG TARGETARCH
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 GOOS=$TARGETOS GOARCH=$TARGETARCH \
    go build -o /out/app ./cmd/app

# The final stage IS the target platform (no --platform override), so the
# runtime image is genuinely arm64/amd64 — only the tiny copy runs "emulated".
FROM alpine:3.20
COPY --from=build /out/app /usr/local/bin/app
ENTRYPOINT ["app"]

BUILDPLATFORM is the host doing the building; TARGETPLATFORM/TARGETARCH/TARGETOS are what this leg of the multi-arch build is for. Because the heavy go build runs on $BUILDPLATFORM (native) and only the final scratch/alpine stage carries the target arch, a cross-compiled Go or Rust image builds at near-native speed even under a single-arch runner with QEMU registered. Interpreted stacks (Python, Node, Ruby) cannot cross-compile their native dependencies this way, which is exactly why they benefit most from native runners or the kubernetes driver.

Cache invalidation reality

BuildKit’s cache key for a step is a hash of its inputs: the instruction, the parent layer’s digest, and (for COPY/ADD) the contents of the files it pulls in. A miss is never random — one of those inputs changed. The three that catch teams:

• Floating base tags. FROM node:20 re-resolves to whatever :20 points at today. When the upstream digest moves, every layer beneath it misses. Pin by digest (FROM node:20@sha256:...) for reproducibility, and bump it deliberately.
• COPY . . too early. If you copy the whole context before installing dependencies, any source edit busts the dependency-install layer. Copy the lockfile, install, then copy the source — the classic ordering rule, and it matters ten times more with mode=max because you are caching every stage.
• Volatile build args. Threading GIT_SHA or a timestamp into an early ARG that a RUN consumes invalidates from that point down. Keep such values in labels or late stages, not in the cacheable build path.

mode=max vs mode=min is the layer-math behind all of this: min exports only the layers that survive into the final image, so a multi-stage build’s deps and build stages are never cached and rebuild cold every run; max exports every intermediate layer of every stage, which is bigger to store but is the only setting that actually caches a layered build end to end.

What a manifest list actually is

A multi-arch “image” is not one image — it is an OCI image index (a manifest list): a small JSON document whose manifests array lists one real image manifest per platform, each with a platform: {architecture, os} field and a digest. docker pull reads the index, matches the caller’s arch, and fetches only that entry. You can see the raw structure:

docker buildx imagetools inspect ghcr.io/kloudvin/api:1.4.0 --raw

Attestations (provenance, SBOM) ride along as referrer manifests — separate objects in the same repo that point back at the image digest via the OCI referrers API. That is why they need type=registry to land: there is nowhere to attach a referrer in the single-platform local image store. Provenance itself is an in-toto statement describing the build (mode=max includes the full set of materials and the invocation), aligning with the SLSA framework so a downstream verifier (slsa-verifier, or a Kyverno/cosign policy at admission) can prove this digest came from that pipeline building that commit before it is allowed to run.

Enterprise scenario

A platform team running ~40 microservices on EKS had a release pipeline that took 34 minutes, dominated by arm64. They had standardized on Graviton nodes for cost, so every image was built linux/amd64,linux/arm64 — but CI ran on amd64 GitHub-hosted runners, so the entire arm64 half went through QEMU. Their Rust and Go services compiled under emulation at roughly a sixth of native speed, and a monorepo change that touched a shared library triggered all 40 builds at once.

The constraint was real cost discipline: they would not run a permanent fleet of self-hosted arm64 runners idling between releases. The fix had three parts. First, they moved arm64 builds onto GitHub’s native ubuntu-24.04-arm hosted runners, eliminating QEMU entirely — the arm64 leg dropped from ~22 minutes to ~4. Second, they converted the per-service shell scripts into a single docker-bake.hcl with a _common base and a matrix over the service list, so the monorepo’s 40 builds ran as one bake group sharing a BuildKit daemon and deduplicating the common base layers. Third, they keyed the registry cache per service and arch and added mode=max, which took the dependency-install stages from cold every run to near-instant.

The merge stayed registry-native — no image ever round-tripped through a runner to be reassembled:

# Final assembly job: one manifest list, zero rebuilds
docker buildx imagetools create \
  --tag $REGISTRY/$SERVICE:$VERSION \
  $REGISTRY/$SERVICE:$VERSION-amd64 \
  $REGISTRY/$SERVICE:$VERSION-arm64

End state: 34 minutes to 9, with SLSA provenance attached at build time via attest = ["type=provenance,mode=max"], and a cache hit rate above 80% on incremental changes. The hidden win was reproducibility — because every target inherited the same base, “works on amd64 but not arm64” drift stopped happening, since both platforms now built from byte-identical instructions.

Practice challenges

Work these in order; each builds on the last. Try before you open the solution. None require a live push — you can validate structure with docker buildx bake <target> --print, which resolves the HCL to JSON without building.

Challenge 1 — Two targets, one base (beginner). Write a docker-bake.hcl with a REGISTRY variable defaulting to ghcr.io/kloudvin, a _base target that sets context = ".", and two targets web and db that inherit it, each with their own dockerfile and a tag of ${REGISTRY}/<name>:dev. Then build only web.

<details><summary>Solution</summary>

variable "REGISTRY" { default = "ghcr.io/kloudvin" }

target "_base" { context = "." }

target "web" {
  inherits   = ["_base"]
  dockerfile = "web/Dockerfile"
  tags       = ["${REGISTRY}/web:dev"]
}

target "db" {
  inherits   = ["_base"]
  dockerfile = "db/Dockerfile"
  tags       = ["${REGISTRY}/db:dev"]
}

docker buildx bake web

Why: inherits puts context in one place; naming the target on the CLI builds just that one. Run docker buildx bake web --print to confirm the resolved tag. </details>

Challenge 2 — Matrix over versions (intermediate). Turn a single app target into a build per Node version — 18, 20, 22 — each tagged ${REGISTRY}/app:node-<version>, passing the version as a NODE_VERSION build arg. Ensure each cell has a unique name.

<details><summary>Solution</summary>

target "app" {
  name       = "app-node-${ver}"
  matrix     = { ver = ["18", "20", "22"] }
  dockerfile = "Dockerfile"
  tags       = ["${REGISTRY}/app:node-${ver}"]
  args       = { NODE_VERSION = ver }
}

docker buildx bake app --print   # shows three resolved targets

Why: the matrix expands one definition into three; the ${ver} in name keeps each cell distinct, which bake requires. </details>

Challenge 3 — Multi-arch with registry cache (advanced). Extend a _common base so every target builds linux/amd64,linux/arm64, reads and writes a registry remote cache at ${REGISTRY}/buildcache:app with mode=max, and pushes to the registry. Write the one command that builds the api target this way.

<details><summary>Solution</summary>

target "_common" {
  context    = "."
  platforms  = ["linux/amd64", "linux/arm64"]
  cache-from = ["type=registry,ref=${REGISTRY}/buildcache:app"]
  cache-to   = ["type=registry,ref=${REGISTRY}/buildcache:app,mode=max"]
  output     = ["type=registry"]
}

REGISTRY=ghcr.io/kloudvin docker buildx bake api

Why: multi-arch and registry cache both require the docker-container (or kubernetes) driver — docker buildx create --driver docker-container --use first. mode=max caches every stage; output=type=registry is what makes attestations and the manifest list land. </details>

Challenge 4 — Split runners, then merge (advanced). You have an amd64 runner and an arm64 runner. Write the two per-arch bake commands (each building only its own platform, tagged with an arch suffix) and the single command that merges :1.0.0-amd64 and :1.0.0-arm64 into :1.0.0 — without rebuilding.

<details><summary>Solution</summary>

# amd64 runner
docker buildx bake api \
  --set "*.platform=linux/amd64" \
  --set "api.tags=ghcr.io/kloudvin/api:1.0.0-amd64" \
  --set "*.output=type=registry"

# arm64 runner
docker buildx bake api \
  --set "*.platform=linux/arm64" \
  --set "api.tags=ghcr.io/kloudvin/api:1.0.0-arm64" \
  --set "*.output=type=registry"

# merge job (any runner)
docker buildx imagetools create \
  --tag ghcr.io/kloudvin/api:1.0.0 \
  ghcr.io/kloudvin/api:1.0.0-amd64 \
  ghcr.io/kloudvin/api:1.0.0-arm64

Why: each runner builds natively (no QEMU); imagetools create assembles the manifest list registry-side, referencing the existing per-arch digests rather than re-pulling or rebuilding them. </details>

Verify

Confirm the whole chain end to end before you trust it in production:

# 1. The resolved plan is what you expect (platforms, tags, cache refs)
docker buildx bake api --print

# 2. The published tag is a true multi-arch manifest list
docker buildx imagetools inspect ghcr.io/kloudvin/api:1.4.0
#    -> expect Platform: linux/amd64 AND linux/arm64

# 3. Pull and run the arm64 variant on an arm64 host (or emulated)
docker run --rm --platform linux/arm64 ghcr.io/kloudvin/api:1.4.0 --version

# 4. Provenance and SBOM attestations are attached
docker buildx imagetools inspect ghcr.io/kloudvin/api:1.4.0 \
  --format '{{json .SBOM}}'

# 5. A no-op rebuild is fully cached
BUILDKIT_PROGRESS=plain docker buildx bake api 2>&1 | grep -c CACHED

If step 5 reports zero, your cache key is unstable — go back to --print and diff the inputs.

Common beginner mistakes

These are misconceptions, not typos — each one is a wrong mental model that produces a confusing symptom.

• “I set --platform linux/amd64,linux/arm64 and got multiple platforms feature is currently not supported.” You are on the default docker driver, which cannot build multi-arch. The right model: multi-arch needs a standalone BuildKit, so docker buildx create --driver docker-container --use first. This is the single most common first wall.
• “My arm64 build takes forever.” You are emulating arm64 under QEMU on an amd64 host, and emulation interprets every instruction. The right model: emulation is a correctness fallback, not a build strategy — put arm64 on a native runner (or cross-compile), and keep QEMU only for interpreted images or a quick local test.
• “Cache works on my laptop but every CI run starts cold.” Your cache is local or inline on an ephemeral runner that is destroyed after the job. The right model: cache must live outside the runner — type=registry (or gha) — and be keyed by something stable (service + arch), not by commit SHA, or the next run can never find it.
• “I turned on cache but my deps stage still rebuilds.” You are on the default mode=min, which only caches the final image’s layers. The right model: multi-stage builds need mode=max to cache intermediate stages — that is the whole point of registry cache for layered images.
• “The image runs on my Mac but the amd64 node says exec format error.” You built a single-arch (arm64) image and it landed on an amd64 host, or you pulled the wrong entry. The right model: a manifest list is what makes one tag work everywhere — build for both platforms and merge, then docker pull auto-selects.
• “My provenance and SBOM disappeared.” You exported with type=docker (a local load), which has nowhere to store referrer manifests. The right model: attestations only survive to a real registry — export type=registry (or type=image,push=true).
• “--load failed on my multi-arch build.” The local Docker image store holds one platform, so it cannot load a two-platform result. The right model: --load a single platform for local testing, and --push (registry) for the real multi-arch artifact.

Glossary

• bake — docker buildx bake, a build orchestrator that reads a declarative file and builds many targets in parallel. The subject of this lesson.
• target — one image build in a bake file; the declarative equivalent of a single docker buildx build.
• group — a named set of targets built together; the default group is what bake builds with no arguments.
• matrix — a target attribute that expands one definition into many builds, one per combination of values.
• inherits — an HCL attribute that copies another target’s fields into this one, so shared config lives in one base target.
• variable — a typed, defaulted input to a bake file, overridable by an environment variable or --set.
• buildx — the Docker CLI plugin that drives BuildKit and provides build, bake, and imagetools.
• BuildKit — the modern build engine underneath buildx: concurrent, cache-aware, and able to emit multi-arch images and attestations.
• driver — how/where a builder runs BuildKit: docker, docker-container, kubernetes, or remote.
• builder — a named build environment created with docker buildx create; it has one driver and one or more nodes.
• multi-arch image — a single tag that carries builds for multiple CPU architectures, so it runs on amd64 and arm64 alike.
• manifest list / OCI image index — the small JSON index that lists one image per platform under one tag; docker pull picks the matching one.
• QEMU / binfmt — the emulation layer that lets an amd64 host build/run arm64 (and vice versa) by interpreting foreign instructions; correct but slow.
• cross-compilation — compiling natively on the build host but emitting code for the target arch (GOARCH=arm64), avoiding QEMU for compiled languages.
• BUILDPLATFORM / TARGETPLATFORM — automatic build args BuildKit sets to the host arch and the requested target arch, used to cross-compile.
• remote cache — build cache stored outside the runner (registry, gha, s3, azblob) so it can be shared across CI runs and machines.
• cache-to / cache-from — where a build writes its cache and where it reads it; the two ends of remote cache.
• mode=max / mode=min — whether the cache exports every intermediate layer (max) or only the final image’s layers (min).
• inline cache — cache metadata embedded in the published image itself; convenient but mode=min only.
• imagetools — the docker buildx imagetools subcommand that inspects and assembles manifests registry-side without rebuilding.
• provenance — an in-toto/SLSA attestation recording how an image was built (source, materials, invocation); mode=max records the full set.
• SBOM — Software Bill of Materials, an inventory of the packages inside an image, emitted as an attestation.
• attestation / referrers API — signed metadata (provenance, SBOM) stored as separate manifests that point back at the image digest.
• digest — the content-addressed sha256:... identity of an image or layer; stable and immutable, unlike a tag.

Checklist

• Builder uses the docker-container driver (not the default docker driver)
• arm64 builds run on native runners; QEMU is reserved for non-compiling images
• Shared config lives in a base target consumed via inherits, not duplicated
• cache-to uses type=registry,mode=max with a stable, per-arch cache ref
• Per-arch images are pushed by tag and merged with imagetools create (no QEMU reassembly)
• attest = ["type=provenance,mode=max", "type=sbom"] is set and exported to a registry
• CI defines concurrency with cancel-in-progress to kill superseded runs
• Cache key is keyed by service and architecture, not by commit SHA
• --print is part of your debugging runbook for cache misses
• No secrets are passed via args (they land in mode=max provenance)