ApplicationSets In Depth: List, Cluster, Git, Matrix, Merge, SCM & Pull-Request Generators

There is a moment on every growing GitOps platform where the deploy repo stops feeling like infrastructure and starts feeling like data entry. You add the fourth cluster and copy-paste twenty Application manifests, changing one cluster name in each. You onboard a new microservice and hand-write the same fifteen-line Application you have written forty times before. The app-of-apps pattern — one parent Application whose source is a directory of child Application manifests — got you this far, but it has an obvious tax: every leaf is a file a human wrote, and the difference between most of those files is a single string.

The ApplicationSet is Argo CD’s answer. It is a controller-managed factory: you describe one Application template, hand it a generator that produces a set of parameters, and the controller stamps out one Application per parameter set — then keeps that set in lockstep with the generator forever. “Deploy this app to every cluster labelled env=prod” becomes a five-line selector that fans across AKS, EKS and GKE and grows itself the moment you register cluster twenty-one. “One app per directory in the mono-repo” becomes a glob that produces a new Application the instant someone commits a new folder. This is the single most leveraged object in Argo CD, and it is also the one that, misconfigured, can create — or delete — hundreds of Applications in one reconcile. This lesson teaches both halves: the power and the safety catch.

Everything here targets Argo CD 2.13+/3.x on Kubernetes 1.29+. Every manifest is a real, schema-correct argoproj.io/v1alpha1 ApplicationSet; every generator field and CLI flag is real. The outputs shown are the representative shape you will see — labelled as such — not a transcript of a specific run.

---

Why this matters

App-of-apps and ApplicationSet are often confused because they solve overlapping problems, but they work at different layers. App-of-apps is a convention: a normal Application that happens to point at a folder of other Application manifests. It has no templating and no intelligence — if you want ten child apps, ten YAML files exist in Git. ApplicationSet is a CRD with its own controller: it holds one template and generates the child Application objects itself, in the cluster, from live inputs. The child Applications it produces are ordinary Applications — the same objects app-of-apps would have contained — so ApplicationSet does not replace the sync engine; it replaces the authoring of the leaves.

The mental model to hold for the whole lesson: an ApplicationSet is for element in generator: create Application(template, element). The generator is the loop’s iterable; the template is the loop body; the output is a set of Application objects the controller owns and continuously reconciles against the generator. When you internalise that, every generator below is just “a different thing to loop over,” and every failure mode is either “the loop produced the wrong number of elements” or “the template rendered a bad Application.”

It is equally important to be precise about what ApplicationSet does not do:

If you have not yet met the base Application object, read Your First Application: the spec, source & destination first — an ApplicationSet’s template.spec is an Application spec, so every field you learned there applies unchanged inside the template.

---

The ApplicationSet CRD: a generator plus a template

Here is a complete, valid ApplicationSet with every structural piece labelled. Read it once top to bottom; the rest of this section dissects it.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: example
  namespace: argocd                     # must live in the Argo CD namespace
spec:
  goTemplate: true                       # use Go text/template (recommended)
  goTemplateOptions: ["missingkey=error"]   # a missing key fails the render loudly
  generators:                            # ONE or more; each yields parameter sets
    - list:
        elements:
          - cluster: dev
            url: https://kubernetes.default.svc
  syncPolicy:                            # APPLICATIONSET-level policy (not the app's)
    preserveResourcesOnDeletion: false   # delete the AppSet => delete generated apps
    applicationsSync: create-update      # create + update generated apps, never delete
  template:                              # the Application blueprint, stamped per element
    metadata:
      name: '{{.cluster}}-guestbook'     # MUST be unique across all elements
      labels:
        env: '{{.cluster}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/argoproj/argocd-example-apps.git
        targetRevision: HEAD
        path: guestbook
      destination:
        server: '{{.url}}'
        namespace: '{{.cluster}}'
      syncPolicy:                        # APPLICATION-level policy (per generated app)
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

The top-level spec has a small, fixed set of fields. Learn these and you can read any ApplicationSet:

The two syncPolicy blocks are not the same block

The single nastiest gotcha in the whole CRD is that syncPolicy appears twice and means two different things. spec.syncPolicy governs how the ApplicationSet controller treats the Applications it generates. spec.template.spec.syncPolicy is the ordinary Application sync policy (automated, syncOptions, retry) that ends up on each generated Application and governs how the application-controller syncs that app to its cluster. Put automated.selfHeal in the wrong one and it does nothing; put preserveResourcesOnDeletion in the template and Kubernetes rejects the Application.

spec.syncPolicy has just two fields, and both are about deletion and modification of the generated Applications themselves:

The applicationsSync modes are worth their own table because each one produces a different surprise:

create-update is the popular “seatbelt” for production fleets: a fat-fingered selector that suddenly matches zero clusters will not delete your apps — it just stops updating them. The trade-off is that legitimately removing a cluster leaves an orphaned Application you must delete by hand.

goTemplate vs the legacy engine — and why goTemplate now

Every parameter a generator emits is substituted into the template. There are two substitution engines, and mixing them up is the second-nastiest gotcha in this lesson.

Enable goTemplate: true and goTemplateOptions: ["missingkey=error"] on every new ApplicationSet. The reason is not style — it is safety. Under the legacy engine a typo like {{.clustr}} (or using {{.cluster}} when goTemplate is off) renders an empty string, and an Application with an empty destination.namespace or a blank name is accepted by the API server and then behaves bizarrely. With missingkey=error the same typo aborts the render, the ApplicationSet reports an error condition, and no broken Application is ever created. Fail loud, not silent.

---

The generators at a glance

Argo CD ships eight generators. They differ only in what they loop over; the template mechanics are identical for all of them.

A ninth, Cluster Decision Resource, defers the cluster list to an external placement CRD; it is covered briefly at the end. The next sections take each generator in turn with a complete manifest and a when-to-use rule. At minimum, study List, Cluster, Git-directories, Matrix and Pull-Request closely — they are the five you will reach for constantly.

---

List generator — the static set

The List generator is the simplest: you write the elements yourself as an array of arbitrary key/value maps, and every key becomes a template parameter. Use it when the set is small, fixed, and known at author time — the canonical case being a handful of environments.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: guestbook-envs
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - list:
        elements:
          - env: dev
            url: https://kubernetes.default.svc
            replicas: "1"
          - env: staging
            url: https://kubernetes.default.svc
            replicas: "2"
          - env: prod
            url: https://kubernetes.default.svc
            replicas: "4"
  template:
    metadata:
      name: 'guestbook-{{.env}}'
      labels:
        env: '{{.env}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/argoproj/argocd-example-apps.git
        targetRevision: HEAD
        path: helm-guestbook
        helm:
          parameters:
            - name: replicaCount
              value: '{{.replicas}}'
      destination:
        server: '{{.url}}'
        namespace: 'guestbook-{{.env}}'
      syncPolicy:
        syncOptions:
          - CreateNamespace=true

This one ApplicationSet produces three Applications — guestbook-dev, guestbook-staging, guestbook-prod — each pointed at the same Helm chart but passing a different replicaCount and landing in its own namespace. The per-env values (replicas) ride along as list keys and feed a Helm parameter.

Numeric-looking values like replicas: "1" are quoted deliberately. Template substitution is textual, and an unquoted 1 can be parsed as an integer that then fails to render into a string field. Quote list values that feed string parameters.

---

Cluster generator — the multi-cluster workhorse

The Cluster generator is where ApplicationSet earns its keep on a real platform. It loops over the clusters registered with Argo CD, optionally filtered by a label selector, and emits one Application per matching cluster. This is how you say “run the monitoring stack on every production cluster” once and have it stay true as the fleet grows — the exact multi-cloud fan-out this course is built around.

Argo CD stores every registered cluster as a Kubernetes Secret in the argocd namespace, labelled argocd.argoproj.io/secret-type: cluster (see multi-cluster registration & cluster Secrets). The Cluster generator’s selector matches labels on those Secrets, so the routing you want is a labelling decision you make once, at registration.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: monitoring-fleet
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - clusters:
        selector:
          matchLabels:
            env: prod
  template:
    metadata:
      name: 'monitoring-{{.name}}'
    spec:
      project: platform
      source:
        repoURL: https://github.com/acme/platform-gitops.git
        targetRevision: main
        path: addons/monitoring
      destination:
        server: '{{.server}}'
        namespace: monitoring
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

Register five prod clusters with env=prod and this ApplicationSet produces five Applications — monitoring-aks-prod-1, monitoring-eks-prod-1, monitoring-gke-prod-1, and so on — each syncing the same add-on to its own cluster. Register a sixth: a sixth Application appears with no edit to any manifest.

The parameters the Cluster generator exposes are fixed and worth memorising:

Registering and labelling clusters on AKS, EKS and GKE

The generator is cloud-agnostic; only getting a kubeconfig context differs per cloud. The pattern is always: fetch the context, then argocd cluster add it with the labels your selector will match. The --label flag is repeatable.

# AKS — fetch the context, then register + label
az aks get-credentials --resource-group rg-prod --name aks-prod-1
argocd cluster add aks-prod-1 --label env=prod --label cloud=azure

# EKS — same shape; only the credential command changes
aws eks update-kubeconfig --name eks-prod-1 --region us-east-1
argocd cluster add arn:aws:eks:us-east-1:111122223333:cluster/eks-prod-1 \
  --label env=prod --label cloud=aws

# GKE — again, only get-credentials is cloud-specific
gcloud container clusters get-credentials gke-prod-1 --region us-central1
argocd cluster add gke_myproject_us-central1_gke-prod-1 \
  --label env=prod --label cloud=gcp

Two cloud-specific notes. First, the get-credentials step wires an exec-plugin into your kubeconfig for ongoing auth — kubelogin for AKS with Entra ID, the aws eks get-token helper for EKS, and gke-gcloud-auth-plugin for GKE — which must be installed where you run argocd cluster add. Second, argocd cluster add then creates an argocd-manager ServiceAccount and ClusterRoleBinding in the target cluster and stores that ServiceAccount’s bearer token in the cluster Secret, so day-to-day reconciliation does not depend on your personal cloud credentials. If you would rather keep cloud IAM in the loop end-to-end (Azure Workload Identity, EKS Pod Identity/IRSA, GKE Workload Identity), that is configured on the Secret’s config.execProviderConfig — a topic for the registration lesson, not the generator.

The in-cluster gotcha

The Cluster generator also implicitly includes the local cluster Argo CD runs on — https://kubernetes.default.svc, known as in-cluster. That surprises people two ways. If you write a selector, the in-cluster is included only if it matches, and the in-cluster has no cluster Secret and no labels by default, so any non-empty selector silently excludes in-cluster. Conversely, an empty selector (selector: {} or no selector at all) matches every registered cluster plus in-cluster, which may deploy your add-on somewhere you did not intend. Decide deliberately: label the in-cluster if you want it in a selector, and never use an empty selector on a generator whose apps should not touch the Argo CD cluster itself.

---

Git generator — one app per directory or file (the mono-repo pattern)

The Git generator loops over the contents of a repository. It has two modes, and they solve two different problems.

Directory mode: one Application per folder

Point it at a glob of directories and it produces one Application per matching directory. This is the mono-repo pattern: a repo where each subfolder is a deployable unit, and adding a folder adds an app for free.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: tenant-apps
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - git:
        repoURL: https://github.com/acme/app-config.git
        revision: main
        directories:
          - path: apps/*
          - path: apps/experimental/*
            exclude: true          # subtract a subtree from the matches above
  template:
    metadata:
      name: '{{.path.basename}}'   # the folder name, e.g. "checkout"
    spec:
      project: tenants
      source:
        repoURL: https://github.com/acme/app-config.git
        targetRevision: main
        path: '{{.path.path}}'     # the full path, e.g. "apps/checkout"
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{.path.basename}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

The directory generator exposes a family of path parameters — these are the ones the brief and the exams care about:

The legacy-engine equivalents are {{path}}, {{path.basename}}, and {{path[0]}}. If you copy a goTemplate example into a legacy ApplicationSet (or forget goTemplate: true), {{.path.basename}} renders empty and every generated app gets a blank name — the exact silent failure missingkey=error exists to prevent.

File mode: parameters from config files

Point the same generator at a glob of files instead, and each matched file is parsed (JSON or YAML) and its contents become the template parameters. Use this when each unit needs richer, structured config than a folder name — a per-cluster registration file, say.

generators:
  - git:
      repoURL: https://github.com/acme/platform-gitops.git
      revision: main
      files:
        - path: "clusters/**/config.json"

Given a clusters/prod/eks-1/config.json like:

{
  "cluster": { "name": "eks-prod-1", "address": "https://eks-prod-1.example.com" },
  "team": "payments"
}

the template can reference the file’s contents directly — {{.cluster.name}}, {{.cluster.address}}, {{.team}} — plus {{.path.basename}} for the folder the file lives in. File mode is the bridge between “GitOps for apps” and “GitOps for cluster configuration”: a reviewer approves a new config.json, and the app appears.

---

Matrix generator — the cartesian product

The Matrix generator combines two child generators and emits one Application for every combination of their outputs. The archetypal use is the platform-team dream: “every add-on on every prod cluster,” expressed as git (directories) × clusters.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: addons-everywhere
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - matrix:
        generators:
          - git:
              repoURL: https://github.com/acme/platform-gitops.git
              revision: main
              directories:
                - path: addons/*
          - clusters:
              selector:
                matchLabels:
                  env: prod
  template:
    metadata:
      name: '{{.path.basename}}-{{.name}}'   # addon name + cluster name = unique
    spec:
      project: platform
      source:
        repoURL: https://github.com/acme/platform-gitops.git
        targetRevision: main
        path: '{{.path.path}}'
      destination:
        server: '{{.server}}'
        namespace: '{{.path.basename}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

With 4 add-on directories and 5 prod clusters, this produces 20 Applications, each named <addon>-<cluster>. Add an add-on folder: 5 new apps. Register a prod cluster: 4 new apps. That multiplicative behaviour is the whole point — and the whole danger.

The multiplication is why the diagram badges this generator in red. Four apps and five clusters is twenty Applications; forty apps and twenty clusters is eight hundred, generated from a single commit and, by default, applied all at once. Two guardrails matter here: always preview with argocd admin applicationset generate before applying (shown in the safety section), and for large fleets add a strategy: RollingSync so a matrix change rolls out cluster-by-cluster instead of as a thundering herd.

Read the diagram left to right: the input (a Git repo’s directories, or the labels on registered cluster Secrets) feeds the ApplicationSet and its generator; the generator produces parameters; the template stamps out one Application per parameter set; and each Application fans out to its target — here the three managed Kubernetes services across clouds — reconciling until all N report Synced/Healthy. Badge 1 marks the generator as the fan-out driver (a wrong selector yields zero apps); badge 3 flags the matrix explosion; badge 4 the goTemplate-or-empty rule; badge 5 the cluster-label selector across AKS/EKS/GKE; and badge 6 the deletion danger the safety section is about.

---

Merge generator — override values by key

Where Matrix multiplies generators, Merge joins them. It takes a base generator plus one or more override generators and merges their parameter sets on a merge key: for each element from the base, if a later generator has an element with the same key value, its fields are layered on top. This is the clean way to say “all prod clusters get the default config, except these two, which get an override.”

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: monitoring-with-overrides
  namespace: argocd
spec:
  goTemplate: true
  generators:
    - merge:
        mergeKeys:
          - server                     # join base + overrides on the cluster URL
        generators:
          - clusters:                  # BASE: every prod cluster
              selector:
                matchLabels:
                  env: prod
          - list:                      # OVERRIDE: one cluster gets longer retention
              elements:
                - server: https://eks-prod-1.example.com
                  retention: "90d"
  template:
    metadata:
      name: 'monitoring-{{.name}}'
    spec:
      project: platform
      source:
        repoURL: https://github.com/acme/platform-gitops.git
        targetRevision: main
        path: addons/monitoring
        helm:
          parameters:
            - name: prometheus.retention
              value: '{{dig "retention" "15d" .}}'   # default 15d unless overridden
      destination:
        server: '{{.server}}'
        namespace: monitoring

Every prod cluster gets the monitoring stack; the one cluster whose server matches the override list gets retention: 90d, and every other cluster falls back to 15d via Sprig’s dig (which safely reads a key with a default). The base generator determines which apps exist; the override generators only modify matching ones — an element that appears solely in an override generator is ignored.

---

SCM Provider generator — one app per repo in an org

The SCM Provider generator scans a whole Git organisation and emits one Application per repository that passes your filters. Use it when your unit of deployment is “a repo” and repos come and go — a platform where every service lives in its own repo and should get an Application automatically once it contains a deploy manifest.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: org-services
  namespace: argocd
spec:
  goTemplate: true
  generators:
    - scmProvider:
        github:
          organization: acme
          tokenRef:
            secretName: github-token
            key: token
        filters:
          - repositoryMatch: ^service-.*      # only repos named service-*
            pathsExist:
              - deploy/kustomization.yaml      # …that actually have a deploy dir
  template:
    metadata:
      name: '{{.repository}}'
    spec:
      project: default
      source:
        repoURL: '{{.url}}'
        targetRevision: '{{.branch}}'
        path: deploy
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{.repository}}'
      syncPolicy:
        syncOptions:
          - CreateNamespace=true

The filters are what keep an org generator from creating an Application for every README-only repo:

Because it holds an org-scoped token, the SCM generator is a security-sensitive object: scope the token to read-only repo metadata, store it in a Secret referenced by tokenRef, and never inline it. Rotate it like any other credential.

---

Pull Request generator — an ephemeral preview per PR

The Pull Request generator is the one that makes reviewers smile: it emits one Application per open pull request, so every PR gets its own live preview environment — and, crucially, when the PR merges or closes, the generator stops producing that element and the Application (and its namespace) is torn down automatically.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: guestbook-previews
  namespace: argocd
spec:
  goTemplate: true
  syncPolicy:
    preserveResourcesOnDeletion: false   # PR close => Application removed => resources pruned
  generators:
    - pullRequest:
        github:
          owner: acme
          repo: guestbook
          tokenRef:
            secretName: github-token
            key: token
          labels:
            - preview                    # only PRs carrying the "preview" label
        requeueAfterSeconds: 120         # poll every 2 min (prefer a webhook in prod)
  template:
    metadata:
      name: 'guestbook-pr-{{.number}}'
    spec:
      project: previews
      source:
        repoURL: https://github.com/acme/guestbook.git
        targetRevision: '{{.head_sha}}'  # the PR's commit, not a branch
        path: manifests
      destination:
        server: https://kubernetes.default.svc
        namespace: 'preview-pr-{{.number}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

The auto-teardown is the whole value, and it hinges on spec.syncPolicy.preserveResourcesOnDeletion: false (the default). When a PR closes, the generator no longer lists it, the controller deletes the guestbook-pr-42 Application, and because that Application carries the resources finalizer, its preview-pr-42 namespace and everything in it are pruned. Set preserveResourcesOnDeletion: true here and you would leak a namespace per merged PR — the opposite of what you want.

The parameters a PR exposes drive the preview’s identity and pinning:

For production, replace polling with a webhook so previews appear within seconds of a PR opening rather than on the next requeueAfterSeconds tick, and put previews in their own permissive AppProject (previews) so a broken PR cannot escape into prod namespaces.

Cluster Decision Resource generator (brief)

The last generator, clusterDecisionResource, hands the cluster-selection decision to an external placement controller — typically Open Cluster Management’s PlacementDecision or a similar duck-typed CRD with a status.decisions list of cluster names. You point the generator at that resource and a small ConfigMap that tells it how to read the resource’s status, and it emits one Application per cluster the external controller decided on:

generators:
  - clusterDecisionResource:
      configMapRef: ocm-placement    # tells the generator how to read the CR
      name: prod-placement           # the PlacementDecision to follow
      requeueAfterSeconds: 180

Reach for it only when a dedicated placement engine already owns “which clusters” — otherwise the Cluster generator’s label selector is simpler and needs no extra controller.

---

Templating in depth: goTemplate, functions, and templatePatch

Every generator hands the template a flat (or nested) set of parameters; the difference between generators is which parameters. This reference table collects the ones you will actually type, so you can stop guessing at param names:

Because goTemplate is real Go text/template with the Sprig function library, you can transform parameters inline instead of forcing every value to be pre-computed in the generator:

templatePatch — when the template alone cannot express it

Sometimes a field must exist for some generated apps and be absent for others — a conditional syncPolicy.automated, an extra annotation on prod only. You cannot express “omit this whole block” with plain substitution, so ApplicationSet provides templatePatch: a Go-template string, rendered per element, whose output is merged onto the generated Application as a strategic patch. It requires goTemplate: true.

spec:
  goTemplate: true
  generators:
    - list:
        elements:
          - env: dev
          - env: prod
  template:
    metadata:
      name: 'app-{{.env}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/acme/app.git
        targetRevision: HEAD
        path: manifests
      destination:
        server: https://kubernetes.default.svc
        namespace: '{{.env}}'
  templatePatch: |
    {{- if ne .env "prod" }}
    spec:
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
    {{- end }}

Non-prod apps get automated self-heal; the prod app renders an empty patch and stays manual-sync. templatePatch is powerful and easy to abuse — its output must be valid YAML/JSON or the render fails — so reach for it only when a conditional structure (not just a conditional value) is genuinely needed.

---

The safety story: preview, delete semantics, and the big danger

Everything that makes ApplicationSet powerful — one object producing many Applications — is exactly what makes it dangerous. A generator does not know you made a mistake. Change a selector to match nothing and, under the default policy, the controller dutifully deletes every Application that selector used to produce, cascading to live workloads. Point a matrix at the wrong glob and it creates hundreds of Applications that all hammer your repo-server at once. The controller is a faithful loop; the blast radius is the size of the generator’s output.

⚠️ The number-one production incident with ApplicationSets is a generator change that silently adds or removes hundreds of elements. Deleting the ApplicationSet, narrowing a selector, renaming a label, or fixing a “harmless” glob can each create or destroy Applications in bulk on the next reconcile. Treat any change to generators as a change that can delete production.

Three controls form the seatbelt, and you should understand all three before you run kubectl apply on anything non-trivial.

1. Preview offline, every time. argocd admin applicationset generate renders the exact Applications an ApplicationSet would produce, from a file, without touching the cluster. This is your dry-run:

# Render the Applications this AppSet would create — nothing is applied
argocd admin applicationset generate addons-everywhere.yaml

# Representative shape: one Application document per generated element
# apiVersion: argoproj.io/v1alpha1
# kind: Application
# metadata:
#   name: monitoring-eks-prod-1
# ...
# ---
# apiVersion: argoproj.io/v1alpha1
# kind: Application
# metadata:
#   name: monitoring-gke-prod-1
# ...

Count the documents. If a four-add-on, five-cluster matrix does not print exactly twenty Applications, your selector or glob is wrong — find out before the controller does.

2. Understand delete semantics. By default the generated Applications are owned by the ApplicationSet (owner references), so deleting the ApplicationSet garbage-collects all of them, and the resources finalizer then prunes their workloads. spec.syncPolicy.preserveResourcesOnDeletion: true breaks that chain and keeps the apps alive when the AppSet is deleted. spec.syncPolicy.applicationsSync: create-update (or create-only) stops the controller from deleting apps at all, so a bad selector cannot prune your fleet — it can only stop updating it.

3. Gate the rollout. For large fan-outs, spec.strategy.type: RollingSync rolls changes across generated apps in labelled steps instead of all at once, so a bad change is caught after the first cluster, not after the eight-hundredth. This is the fleet-scale safety valve.

When you are unsure which generator even fits the problem, this decision table shortcuts the choice:

And the danger table — the ways a generator turns into an outage:

---

Hands-on lab

You will build three ApplicationSets on a free local cluster — a List, a Git-directory, and a Cluster generator — apply them, watch the Applications they generate, and tear them down understanding exactly what deletion prunes. Nothing here bills: it is all a local kind cluster and public repos.

Prerequisites. A local cluster with Argo CD installed and the argocd CLI logged in — the state you reach at the end of the install lesson:

# A throwaway local cluster
kind create cluster --name argocd-lab

# Argo CD (namespace + core install)
kubectl create namespace argocd
kubectl apply -n argocd \
  -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

# Wait for the controllers, then confirm the ApplicationSet controller is up
kubectl -n argocd rollout status deploy/argocd-applicationset-controller

What just happened: the standard install includes the applicationset-controller alongside the application-controller, repo-server, and API server. If that Deployment is Available, ApplicationSets will reconcile.

Step 1 — A List generator across three environments. Save this as lab-list.yaml:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: lab-guestbook-envs
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - list:
        elements:
          - env: dev
            replicas: "1"
          - env: staging
            replicas: "2"
          - env: prod
            replicas: "3"
  template:
    metadata:
      name: 'guestbook-{{.env}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/argoproj/argocd-example-apps.git
        targetRevision: HEAD
        path: helm-guestbook
        helm:
          parameters:
            - name: replicaCount
              value: '{{.replicas}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: 'guestbook-{{.env}}'
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true

Preview it before applying, then apply:

# Dry-run: see exactly three Applications, nothing applied
argocd admin applicationset generate lab-list.yaml | grep '^  name:'
#   name: guestbook-dev
#   name: guestbook-staging
#   name: guestbook-prod   (representative — three documents)

kubectl apply -f lab-list.yaml

What just happened: one ApplicationSet became three Applications, each a different replicaCount in its own namespace, from a single 25-line file.

Step 2 — A Git-directory generator, one app per folder. Save as lab-git.yaml. The *guestbook glob matches exactly the three deployable guestbook folders in the public example repo:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: lab-git-dirs
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - git:
        repoURL: https://github.com/argoproj/argocd-example-apps.git
        revision: HEAD
        directories:
          - path: '*guestbook'      # guestbook, helm-guestbook, kustomize-guestbook
  template:
    metadata:
      name: 'dir-{{.path.basename}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/argoproj/argocd-example-apps.git
        targetRevision: HEAD
        path: '{{.path.path}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: 'lab-{{.path.basename}}'
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true

kubectl apply -f lab-git.yaml

What just happened: the generator scanned the repo, found three directories ending in guestbook, and produced dir-guestbook, dir-helm-guestbook, and dir-kustomize-guestbook — Argo CD auto-detected plain-YAML, Helm and Kustomize in each. Commit a fourth matching folder and a fourth app would appear on the next scan.

Step 3 — A Cluster generator fanning to env=prod. On a single kind cluster you have only the in-cluster, which is unlabelled, so first label it, then apply a selector-based ApplicationSet. Save as lab-cluster.yaml:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: lab-cluster-fanout
  namespace: argocd
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - clusters:
        selector:
          matchLabels:
            env: prod
  template:
    metadata:
      name: 'fleet-{{.nameNormalized}}'
    spec:
      project: default
      source:
        repoURL: https://github.com/argoproj/argocd-example-apps.git
        targetRevision: HEAD
        path: guestbook
      destination:
        server: '{{.server}}'
        namespace: fleet-guestbook
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true

# The in-cluster is represented by a Secret named "in-cluster" only after you
# register or label it. Label it so a selector can match it:
kubectl -n argocd label secret \
  $(kubectl -n argocd get secret -l argocd.argoproj.io/secret-type=cluster \
     -o name | head -n1) env=prod --overwrite 2>/dev/null \
  || echo "No cluster Secret yet — see note below"

kubectl apply -f lab-cluster.yaml

What just happened: the Cluster generator matched the one Secret labelled env=prod and produced a single Application, fleet-in-cluster. On a real platform with AKS, EKS and GKE clusters each labelled env=prod, the same ApplicationSet would produce one Application per cloud — the multi-cluster fan-out, with no manifest edits. If your lab has no cluster Secret at all yet, that is expected on a fresh single-node install; the multi-cluster registration lesson shows how argocd cluster add creates and labels them.

Step 4 — See what you built.

kubectl get applicationset -n argocd
# NAME                   AGE
# lab-guestbook-envs     2m
# lab-git-dirs           1m
# lab-cluster-fanout     30s

kubectl get applications -n argocd
# NAME                     SYNC STATUS   HEALTH STATUS
# guestbook-dev            Synced        Healthy
# guestbook-staging        Synced        Healthy
# guestbook-prod           Synced        Healthy
# dir-guestbook            Synced        Healthy
# dir-helm-guestbook       Synced        Healthy
# dir-kustomize-guestbook  Synced        Healthy
# fleet-in-cluster         Synced        Healthy      (representative shape)

# Which ApplicationSet owns a given Application?
kubectl get application guestbook-dev -n argocd \
  -o jsonpath='{.metadata.ownerReferences[0].kind}/{.metadata.ownerReferences[0].name}{"\n"}'
# ApplicationSet/lab-guestbook-envs

What just happened: three ApplicationSets produced seven Applications between them, and each Application carries an ownerReference back to the ApplicationSet that generated it. That owner reference is the mechanism behind the deletion semantics you are about to see.

Step 5 — Teardown, with the delete semantics made explicit.

# Deleting an ApplicationSet deletes the Applications it owns,
# and (because they carry the resources finalizer) their workloads too.
kubectl delete applicationset lab-guestbook-envs -n argocd
kubectl delete applicationset lab-git-dirs -n argocd
kubectl delete applicationset lab-cluster-fanout -n argocd

# Confirm the generated Applications are gone
kubectl get applications -n argocd
# No resources found in argocd namespace.

# Remove the whole cluster
kind delete cluster --name argocd-lab

⚠️ That kubectl delete applicationset is the exact command that, in production, prunes every generated app and its running workloads. If you ever need to delete an ApplicationSet but keep its apps running, set spec.syncPolicy.preserveResourcesOnDeletion: true first, apply, and only then delete — or the generated Applications go with it.

---

Common mistakes and troubleshooting

ApplicationSet failures cluster into “the generator produced the wrong set” and “the template rendered a bad Application.” This table covers the ones you will actually hit; the prose after it dwells on the three that cost the most hours.

Zero apps is almost always the selector or the glob. When an ApplicationSet produces nothing, resist editing the template — the template never ran, because the generator emitted no elements. Run argocd admin applicationset generate on the file: if it prints no documents, the generator is the problem. For a Cluster generator, list your labelled Secrets (kubectl -n argocd get secret -l argocd.argoproj.io/secret-type=cluster --show-labels) and confirm one actually carries the selector’s label. For a Git generator, remember the glob is relative to the repo root and that apps/* matches directories inside apps, not apps itself. kubectl describe applicationset <name> surfaces the controller’s ErrorOccurred and ParametersGenerated conditions, which name the failure directly.

The goTemplate/legacy mismatch is invisible without missingkey=error. Under the legacy engine, {{.cluster}} is not a valid reference, so it renders as the literal empty string — and an Application with name: "" or namespace: "" is often accepted by the API server and then misbehaves in confusing ways (deploys to default, or collides with another empty-named app). The fix is preventive: set goTemplate: true and goTemplateOptions: ["missingkey=error"] on every ApplicationSet, so a wrong reference aborts the render and shows up as an error condition instead of a silently broken app. If you inherit an ApplicationSet with mysterious empty fields, check goTemplate first.

Delete blast radius is a property of ownership, not intent. The controller does not distinguish “I meant to delete this” from “my selector accidentally matches nothing now” — in both cases the generated set shrank, so the corresponding Applications are removed. Before any change to a generators block on a production ApplicationSet, preview the new output and diff the app count against the current kubectl get applications. If the count drops unexpectedly, stop: you are about to prune live workloads. applicationsSync: create-update is the seatbelt that turns “delete my fleet” into merely “stop updating my fleet,” and it is cheap insurance on anything that matters.

---

Cheat-sheet

Generator selection at a glance:

Templating quick-reference (assume goTemplate: true):

Commands and CRD fields:

---

Interview and exam questions

Q: What problem does ApplicationSet solve that app-of-apps does not? A: App-of-apps still requires a hand-written child Application manifest per app — pure boilerplate when the only difference is a cluster name or a values file. ApplicationSet templates the children from a generator, so adding the Nth app or cluster requires no new manifest: the controller generates and reconciles the leaves for you.

Q: An ApplicationSet has two syncPolicy blocks. What is the difference? A: spec.syncPolicy is ApplicationSet-level and controls how the controller treats the generated Applications — preserveResourcesOnDeletion and applicationsSync. spec.template.spec.syncPolicy is the ordinary Application sync policy (automated, syncOptions, retry) that ends up on each generated app and controls how it syncs to its cluster. They are not interchangeable.

Q: Why enable goTemplate: true and missingkey=error on every ApplicationSet? A: The legacy fasttemplate engine cannot do conditionals or safe nested lookups, and a mistyped or missing parameter renders as an empty string — producing a broken-but-accepted Application. goTemplate gives you Go templating plus the Sprig functions, and missingkey=error turns a bad reference into a render failure and an error condition instead of a silent empty field.

Q: How does the Cluster generator decide which clusters to target, and what is the in-cluster gotcha? A: It matches a label selector against the labels on each registered cluster’s Secret (argocd.argoproj.io/secret-type: cluster). The gotcha is the local in-cluster: it has no Secret or labels by default, so any non-empty selector excludes it, while an empty selector includes it plus every registered cluster — potentially deploying somewhere you did not intend.

Q: You register three prod clusters on AKS, EKS and GKE and label them env=prod. Sketch the ApplicationSet that runs one add-on on all three. A: A Cluster generator with selector.matchLabels.env: prod, and a template whose metadata.name is addon-{{.name}} and destination.server is {{.server}}. It produces one Application per matching cluster and grows automatically when a fourth env=prod cluster is registered — no manifest edit.

Q: What is the difference between the Matrix and Merge generators? A: Matrix produces the cartesian product of two child generators — one app per combination (apps × clusters). Merge joins generators on a merge key and layers later generators’ fields onto matching base elements — same set of apps, with per-item overrides. Matrix multiplies the count; Merge does not.

Q: How does the Pull Request generator give you preview environments, and how do they get cleaned up? A: It emits one Application per open PR, typically pinning source.targetRevision to {{.head_sha}} and namespacing by {{.number}}. When the PR closes, the generator stops listing it, the controller deletes the Application, and — because preserveResourcesOnDeletion is false (the default) and the app carries the resources finalizer — the preview namespace and its workloads are pruned automatically.

Q: A colleague narrowed a Cluster generator’s selector and half the fleet’s apps disappeared. What happened and how do you prevent it? A: The generated set shrank, so under the default policy the controller deleted the Applications that no longer matched — cascading to their workloads. Prevent it by previewing with argocd admin applicationset generate and diffing the app count before applying, and by setting applicationsSync: create-update so the controller never deletes generated apps, only stops updating them.

Q: How do you safely preview what an ApplicationSet will create before applying it? A: argocd admin applicationset generate <file> renders the exact Applications the ApplicationSet would produce, from the file, without touching the cluster. Count the documents and check the names; if the number is not what you expect, the selector or glob is wrong.

Q: What does applicationsSync: create-update actually change? A: It permits the controller to create and update generated Applications but never delete them. Removing an element (or a selector matching nothing) will not prune the corresponding app — protecting production from accidental fan-in — at the cost of leaving orphaned Applications you must delete by hand when a removal is intentional.

Q: When would you use the Git files generator over the directories generator? A: When each unit needs structured configuration richer than a folder name. Directory mode gives you path params only; file mode parses each matched JSON/YAML file and exposes its contents as parameters ({{.cluster.name}}, {{.team}}), which is ideal for config-driven fleets where a reviewer adds a config.json to onboard a cluster or tenant.

Q: What is templatePatch for, and what is its constraint? A: It applies a Go-template-rendered patch onto each generated Application after the main template renders — used to add or omit whole blocks conditionally (e.g. automated sync on non-prod only), which plain substitution cannot express. It requires goTemplate: true, and its rendered output must be valid YAML/JSON or the render fails.

---

Key takeaways

• An ApplicationSet is a loop: a generator produces parameter sets, the template stamps one Application per set, and the controller keeps that set reconciled with the generator forever. App-of-apps authors leaves by hand; ApplicationSet generates them.
• Eight generators, one mechanism. List (static set), Cluster (fleet by label — the AKS/EKS/GKE workhorse), Git directories (one app per folder), Git files (params from config), Matrix (cartesian product), Merge (override by key), SCM Provider (one app per org repo), and Pull Request (ephemeral preview per PR). Pick by what you loop over.
• goTemplate: true + goTemplateOptions: ["missingkey=error"] on every ApplicationSet. The legacy engine renders typos as empty strings and builds silently-broken Applications; goTemplate fails loud and unlocks conditionals, Sprig functions, and templatePatch.
• The two syncPolicy blocks mean different things. spec.syncPolicy (preserveResourcesOnDeletion, applicationsSync) governs the generated Applications; spec.template.spec.syncPolicy governs how each app syncs to its cluster.
• A bad generator can create or delete hundreds of apps. Always preview with argocd admin applicationset generate, diff the app count, guard fleets with applicationsSync: create-update, and gate large fan-outs with strategy: RollingSync.
• Deleting the ApplicationSet prunes its generated apps and their workloads unless preserveResourcesOnDeletion: true — the same owner-reference chain that makes PR previews self-clean makes an accidental kubectl delete applicationset an outage.
• Label clusters once, route forever. Register each cluster with argocd cluster add CTX --label env=prod --label cloud=…; the Cluster generator’s selector then fans apps across clouds and grows itself as the fleet grows.