Sync Waves & Resource Hooks: Ordering, PreSync/Sync/PostSync/SyncFail & Jobs

Here is a problem that does not exist until, one day, it very much does. You point an Argo CD Application at a directory, hit sync, and Argo CD applies everything in it at once: the namespace, the ConfigMap, the Deployment, the Service, the Job that seeds your database. Ninety percent of the time this is exactly what you want — Kubernetes is declarative, controllers converge, order sorts itself out. Then you add one resource that has a prerequisite, and the whole thing falls over. The custom resource lands before its CRD is registered and the apply is rejected. The app pods start before the schema migration has run and they crash on a missing column. The ResourceQuota applies before the namespace exists and errors out. Nothing is wrong with your manifests — they are just being applied in the wrong order.

Argo CD gives you two mechanisms to fix this, and they are the entire subject of this lesson. Sync waves let you number your resources into ordered groups that apply one after another, with Argo CD waiting for each group to become Healthy before starting the next. Resource hooks let you run a resource — almost always a Job — at a defined point relative to the sync: before it (PreSync), during it (Sync), after everything is healthy (PostSync), or only when the sync fails (SyncFail). Used together they turn “apply a pile of YAML” into “run an ordered, gated deployment pipeline,” all declared in Git.

This is a cloud-neutral topic — waves and hooks behave identically on AKS, EKS, GKE, or a laptop kind cluster, because they are pure Argo CD sync mechanics with nothing cloud-specific in them. (The one real edge — a migration Job that needs credentials to reach a managed database — gets a short section of its own.) So the lab runs on a free local cluster and everything you learn transfers everywhere. We build the mental model, go phase by phase and policy by policy with real annotated manifests, then assemble the whole ordered deploy in the lab and watch the phases run in sequence.

---

Why this matters

Argo CD is a reconciliation engine: it renders your desired state from Git and drives the cluster to match it. For a single self-contained app, “apply it all and let Kubernetes converge” is the right model and you should not reach for ordering at all — adding waves you do not need is its own kind of bug. Ordering matters only when one resource cannot be healthy until another one already is, and that dependency is invisible to a plain kubectl apply -f ..

Three dependencies show up again and again, and every platform engineer meets all three eventually:

Notice what these have in common: the failure is not a typo in your YAML, and re-running the sync sometimes fixes it (the CRD registered on the first pass, so the CR applies on the second) and sometimes does not (the migration will never run after the pods, because Argo CD applied them together). “Just sync again” is not an ordering strategy — it is a coin flip that happens to land right when Kubernetes’ own retries paper over the gap.

The mental model to hold for the rest of this lesson: a sync is not one instant — it is a sequence you can shape. By default that sequence is Argo CD’s built-in ordering (namespaces and CRDs first, then everything else, roughly at once). Sync waves let you insert your own ordered checkpoints into it. Hooks let you attach run-once work — migrations, tests, cleanup — to the boundaries of it. Get these two tools right and a large chunk of “flaky deploy” tickets simply stop happening.

One sentence to anchor everything below: waves order the resources within a sync; hooks attach extra work to the phases of a sync. They compose — a hook can also carry a wave — but they answer different questions. Keep them as two words in your head.

---

Two mechanisms: sync waves and resource hooks

Before the details, get the shapes of the two tools clear, because beginners routinely reach for the wrong one. Both are driven by annotations on your Kubernetes resources — no new CRD, no controller flag, just metadata Argo CD reads at sync time.

The single most useful distinction: a normal resource with a sync-wave is still your permanent desired state — it lives in the cluster, self-heals, gets pruned when you delete it from Git. A hook is transient work — a migration that runs and finishes, a test that passes and is deleted. If the thing should keep running, it is a wave’d resource, not a hook. If the thing should run once at a boundary and go away, it is a hook.

When do you use which? Most real deploys use both, but the trigger for each is different:

Here is the whole model as one picture. Read it left to right: the application-controller starts a sync operation and consults phases (PreSync → Sync → PostSync) and, within each phase, waves. The PreSync migration Job runs and must succeed; then the Sync phase applies wave 0 (ConfigMap + Deployment) and waits for it to be Healthy before wave 1 (the dependent app); once everything is Healthy the PostSync smoke-test Job runs; and if the operation fails anywhere, the SyncFail hook fires instead.

The badges mark the ideas worth memorizing: waves order resources within a phase, lowest first (1); PreSync runs before the Sync phase and, because it runs on every sync, must be idempotent (2); waves are health-gated — wave 0 must be Healthy before wave 1 starts, which is how a database precedes its app (3); a resource that never becomes Healthy stalls the sync at that wave forever (4); PostSync runs only after the whole Sync phase is Healthy, so it is where smoke tests belong (5); and SyncFail fires only when the operation actually fails, with delete policies keeping hook Jobs from piling up (6). If you understand only this diagram, you already understand ordered deploys better than most people running Argo CD.

The rest of the lesson makes each of those six ideas concrete, then wires the whole pipeline together in the lab. The health-gating in badge 3 leans directly on the Sync Status & Health Assessment model — a wave “becomes Healthy” using exactly the per-kind health checks that lesson taught; if that idea is fuzzy, skim it first, because health is the gate that makes waves work.

---

Sync waves: ordering resources inside one sync

A sync wave is a number you attach to a resource with the annotation:

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-1"   # note: a STRING, quoted

The rules are few and precise:

That fifth rule — the health gate — is what makes waves more than cosmetic sorting. Argo CD does not merely apply wave −1 before wave 0; it applies wave −1, then blocks, watching health, and only proceeds to wave 0 once every resource in wave −1 reports Healthy. This is precisely how you express “the database must be up, not just created, before the app rolls.” A StatefulSet in wave 0 and a Deployment in wave 1 means the app pods do not even get applied until the database reports its replicas ready.

The full ordering: phase, then wave, then kind, then name

Waves are one term in a four-part sort key that Argo CD uses for every sync. Knowing all four explains behavior that otherwise looks random:

Two consequences fall straight out of this table. First, the phase always dominates the wave. A PreSync hook runs before every Sync-phase resource, no matter what wave numbers are involved — a PreSync hook in wave 5 still runs before a Sync resource in wave −5, because PreSync is a whole phase earlier. The wave on a hook only orders it against other hooks in the same phase. Second, even with no waves at all, Argo CD is not applying purely at random — the built-in kind ordering already puts Namespaces and CRDs early, which is why some ordering problems seem to fix themselves.

The implicit ordering, and why it is not enough

Argo CD’s built-in kind order handles the easy cases for free:

So a namespace and the ConfigMap inside it usually apply in the right order without a single wave, because the kind order already favors them. The trouble is that the kind order is fixed and generic — it knows nothing about your dependencies. It cannot know that your Deployment needs a migration first (both are “later” kinds), or that this operator must be Healthy before that custom resource. And the classic CRD-before-CR case is only partly solved: the kind order applies the CRD before the CR, but on a first-ever sync the CR’s dry-run can still fail because the API server has not finished establishing the freshly-applied CRD. That is where waves (CRD in wave −1, CR in wave 0) or the SkipDryRunOnMissingResource=true sync option make the ordering explicit and reliable rather than hopeful.

A pragmatic wave layout that scales — treat it as a starting convention, not law:

Waves do not cross Applications. This is the single most common wave misconception, and it is worth stating twice. sync-wave orders resources inside one Application’s sync. If your operator lives in Application A and your custom resources live in Application B, wave numbers on B’s resources will never wait for A — the two Applications sync independently. To order across Applications you put the waves on the child Application objects in an app-of-apps, or gate on health between stacked ApplicationSets. Do not expect two Applications to honor each other’s wave numbers; they do not.

---

Resource hooks: the four phases (plus Skip)

A hook is any resource — overwhelmingly a Job — annotated with a phase, telling Argo CD to run it at that point in the sync operation instead of treating it as permanent desired state:

metadata:
  annotations:
    argocd.argoproj.io/hook: PreSync

There are six phase values. The four in the lesson title are the ones you will use daily; the other two round out the picture:

The mental model for the run phases is a timeline: PreSync happens, and must succeed, before anything in your app changes. Sync applies your app. PostSync happens only after your app is fully Healthy. SyncFail is the catch that fires if any of that errors. A hook is not magic — a PreSync Job is a perfectly ordinary Kubernetes Job; the annotation just tells Argo CD to run it, wait for it to complete, and only then proceed to the Sync phase.

Three properties of hooks trip people up, so name them now:

• Hooks run on every sync, not just the first. A PreSync migration runs again on the next sync, and the next. This is why idempotency is non-negotiable (next section). It is not “run once ever” — it is “run once per sync.”
• A hook is identified purely by the annotation. Remove the annotation and the same Job becomes a permanent Sync-phase resource that Argo CD will try to keep alive and prune. Add it and the Job becomes transient hook work. The manifest body is identical.
• You can put a resource in multiple phases with a comma-separated list: argocd.argoproj.io/hook: PreSync,PostSync runs the same manifest in both phases. Useful for a check you want both before and after.

A hook can technically be any kind, but in practice:

Because a hook Job reports completion through its own Complete/Failed conditions, Argo CD knows whether the phase succeeded by reading the Job’s status — the same health assessment machinery that governs everything else. A PreSync Job that never completes leaves the sync stuck in the PreSync phase; a PostSync Job that fails flips the whole sync to failed. Both are covered in troubleshooting.

Here is a complete, real PreSync hook — a schema migration Job:

apiVersion: batch/v1
kind: Job
metadata:
  name: db-migrate
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
    argocd.argoproj.io/sync-wave: "-1"
spec:
  backoffLimit: 2                 # retry a couple of times before failing the hook
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: registry.example.com/myapp-migrations:1.8.0
          command: ["/bin/sh", "-c"]
          args:
            - |
              # A real migration tool tracks which migrations have already run,
              # so re-running this on every sync is safe (idempotent).
              migrate -path /migrations -database "$DATABASE_URL" up
          envFrom:
            - secretRef:
                name: db-credentials   # provided out-of-band; never in this manifest

And a SyncFail hook that cleans up when a sync goes wrong:

apiVersion: batch/v1
kind: Job
metadata:
  name: rollback-canary
  annotations:
    argocd.argoproj.io/hook: SyncFail
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  backoffLimit: 1
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: cleanup
          image: bitnami/kubectl:1.31
          command: ["/bin/sh", "-c"]
          args:
            - |
              echo "Sync failed — removing the half-applied canary."
              kubectl delete deployment myapp-canary --ignore-not-found

Note that neither manifest contains a plaintext secret — the migration credentials come from a Secret (db-credentials) created out-of-band by Sealed Secrets, External Secrets, or your cloud’s secret store, never committed in the Job. That rule does not relax just because a resource is a hook.

---

Hook delete policies: taming the Job pileup

A hook Job runs, finishes, and then… sits there. Run ten syncs and, unless you say otherwise, you can accumulate ten completed db-migrate Jobs (and their pods) cluttering the namespace. The hook delete policy controls when Argo CD cleans a hook resource up:

metadata:
  annotations:
    argocd.argoproj.io/hook-delete-policy: HookSucceeded

There are three policies, and choosing the right one per hook is the difference between a tidy namespace and a graveyard of finished Jobs:

The subtlety that catches everyone: the default policy is BeforeHookCreation. If you specify nothing, Argo CD deletes the previous hook instance right before creating the new one on the next sync. That keeps at most one instance around — but it means a completed hook Job stays visible until the next sync runs, and it never cleans up promptly on success. People expect finished hooks to vanish and are surprised to find yesterday’s db-migrate still sitting Completed in the namespace.

The real pileup happens in one specific case, and it is worth understanding exactly:

The takeaway is a rule of thumb: for a hook you do not need to inspect on success, set HookSucceeded. For a hook whose failures you want to debug (almost all of them), do not set HookFailed — let the failed Job stick around so you can kubectl logs it. You can combine policies with a comma (HookSucceeded,BeforeHookCreation), but for most migrations and tests a plain HookSucceeded is exactly right: it cleans up the happy path and preserves the evidence on failure.

---

Waves, hooks, and the classic patterns

Waves and hooks compose. A hook is placed in a phase by its hook annotation, and ordered within that phase by its sync-wave annotation. So a PreSync Job with sync-wave: "-1" runs in the PreSync phase (before any Sync-phase resource) and, if there were other PreSync hooks, would run before those with higher waves. In practice you use the wave on hooks mostly to order multiple hooks in the same phase — a backup PreSync (wave −2) before a migration PreSync (wave −1), say.

Three patterns account for the vast majority of real hook usage. Learn them as recipes.

Pattern 1 — the schema migration (PreSync Job, and the idempotency rule)

The archetype. Your new app version needs a database change; the change must land before the new pods, or they crash. A PreSync Job runs the migration; the Sync phase then rolls the app.

The rule that makes or breaks this pattern: the migration must be idempotent, because the PreSync hook runs on every single sync. Auto-sync on a new commit, a manual sync, or a self-heal — each is a sync, and each re-runs your PreSync migration. A naive CREATE TABLE orders (...) succeeds the first time and then fails on the second sync with “table already exists,” which fails the PreSync hook, which aborts the whole sync. Real migrations are idempotent by construction:

Use a real migration tool if you possibly can — hand-rolled IF NOT EXISTS scripts drift out of sync with reality fast. The point to internalize is that Argo CD gives you no “run once ever” hook; every hook is “run once per sync,” so idempotency is the price of admission for PreSync work.

Pattern 2 — the post-deploy smoke test (PostSync Job)

After the deploy is fully Healthy, prove it actually serves traffic before you call the sync a success. A PostSync Job curls the service and exits non-zero if the response is wrong:

apiVersion: batch/v1
kind: Job
metadata:
  name: smoke-test
  annotations:
    argocd.argoproj.io/hook: PostSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  backoffLimit: 3            # tolerate a little startup jitter before failing
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: smoke
          image: curlimages/curl:8.10.1
          command: ["/bin/sh", "-c"]
          args:
            - |
              # Same namespace, so the short service name resolves.
              curl -sf --retry 5 --retry-delay 3 http://web/ | grep -q "ordered deploy OK"

The power of this pattern is that a failing PostSync fails the whole Application — the sync goes to a failed state, which you can alert on, and which triggers any SyncFail hook. The danger is the flip side: a flaky smoke test fails perfectly good deploys. If your test hits a still-warming cache or a dependency with its own startup lag, use --retry, a generous backoffLimit, and assert on something stable. A smoke test that fails 5% of the time will erode trust in the whole pipeline faster than no test at all.

Pattern 3 — the one-time data seed (guarded hook)

Seeding reference data (country codes, default roles) is “run once ever,” which Argo CD does not offer — so you build it out of a “run once per sync” hook plus a guard. A PreSync or Sync-phase Job that first checks whether the data exists and exits early if it does:

args:
  - |
    if [ "$(count-rows countries)" -gt 0 ]; then
      echo "Reference data already present — nothing to seed."; exit 0
    fi
    seed-reference-data

Same idempotency discipline as the migration: the seed must be safe to re-run because it will be re-run.

Is any of this cloud-specific?

Sync waves and hooks are pure Argo CD sync mechanics — identical on AKS, EKS, GKE, and a local kind cluster. There is no per-cloud behavior in the ordering itself. The one genuine edge appears only when a hook needs to reach a managed dependency: a PreSync migration Job connecting to a managed database needs an identity and credentials, and that is cloud-specific — not the hook, but how the Job’s ServiceAccount authenticates.

The hook manifest is the same on all three; only the ServiceAccount annotation and the secret source differ, and those are covered in the identity and secrets lessons. Everywhere else in this topic, “cloud-neutral” is the honest and complete answer.

---

From Helm hooks to Argo CD hooks

If you deploy Helm charts through Argo CD, you inherit a translation you should understand, because charts carry their own lifecycle hooks (helm.sh/hook) and Argo CD never runs helm install. As the Helm integration lesson explains, the repo-server runs helm template to render plain YAML — so Argo CD re-expresses Helm’s install/upgrade hooks as its own sync phases at render time:

The ordering and cleanup annotations map across too, which is what makes the translation usable rather than merely approximate:

The practical rule: a chart whose hooks do install/upgrade work (migrations, seeds) generally just works on Argo CD, because those map cleanly to PreSync/PostSync and hook-weight behaves like a wave. A chart that leans on helm test, delete/rollback hooks, or .Release.IsUpgrade branching needs attention — you either re-express the behavior as native Argo CD hooks yourself or accept that those paths never fire. If a chart’s hook Job mysteriously never runs, “it is a delete/rollback/test hook” is the first thing to check.

---

Hands-on lab

Time to build the whole ordered pipeline and watch the phases run in sequence. This lab runs on a free local cluster (kind or minikube) with Argo CD installed — nothing here bills, and everything is cloud-neutral. It assumes you can create an Argo CD Application from a Git repo, exactly as in Your First Application. We deploy: a PreSync migration Job (wave −1, a fake migration that sleeps then succeeds, cleaned up with HookSucceeded), a wave 0 ConfigMap + Deployment + Service, and a PostSync smoke-test Job that curls the service. Then we sync and watch PreSync → Sync → PostSync run in order.

The outputs below are representative shapes, not a transcript from your exact cluster — revisions, pod suffixes, and timings will differ. The ordering and the states are what matter. There is no live cluster behind this text; the manifests are schema-correct and the output is what Argo CD produces for this shape of app.

Step 0 — Lay out the repo. Put four files in a directory (ordered-demo/) in a Git repo Argo CD can read:

ordered-demo/
  01-presync-migrate.yaml    # PreSync hook, wave -1
  02-config.yaml             # ConfigMap, wave 0
  03-deploy.yaml             # Deployment + Service, wave 0
  04-postsync-smoke.yaml     # PostSync hook

01-presync-migrate.yaml — the migration hook. It sleeps 5 seconds to simulate a real migration, then exits 0:

apiVersion: batch/v1
kind: Job
metadata:
  name: db-migrate
  annotations:
    argocd.argoproj.io/hook: PreSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
    argocd.argoproj.io/sync-wave: "-1"
spec:
  backoffLimit: 2
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: busybox:1.36
          command: ["/bin/sh", "-c"]
          args:
            - |
              echo "PreSync: running idempotent schema migration..."
              sleep 5
              echo "PreSync: migration complete."

02-config.yaml — the ConfigMap the app serves, wave 0:

apiVersion: v1
kind: ConfigMap
metadata:
  name: web-config
  annotations:
    argocd.argoproj.io/sync-wave: "0"
data:
  index.html: |
    <h1>KloudVin demo — ordered deploy OK</h1>

03-deploy.yaml — the Deployment (mounting that ConfigMap) and its Service, wave 0:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: web
          image: nginx:1.27-alpine
          ports:
            - containerPort: 80
          volumeMounts:
            - name: html
              mountPath: /usr/share/nginx/html
      volumes:
        - name: html
          configMap:
            name: web-config
---
apiVersion: v1
kind: Service
metadata:
  name: web
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80

04-postsync-smoke.yaml — the smoke test, which curls the service after everything is Healthy:

apiVersion: batch/v1
kind: Job
metadata:
  name: smoke-test
  annotations:
    argocd.argoproj.io/hook: PostSync
    argocd.argoproj.io/hook-delete-policy: HookSucceeded
spec:
  backoffLimit: 3
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: smoke
          image: curlimages/curl:8.10.1
          command: ["/bin/sh", "-c"]
          args:
            - |
              echo "PostSync: smoke-testing the web service..."
              curl -sf --retry 5 --retry-delay 3 http://web/ | grep -q "ordered deploy OK"
              echo "PostSync: smoke test passed."

What just happened: four manifests, three ordering annotations. The migration is a PreSync hook so it runs before the app; the ConfigMap and Deployment share wave 0 so they apply together in the Sync phase; the smoke test is PostSync so it runs only after the Deployment is Healthy. Commit and push all four.

Step 1 — Create the Application and sync it.

# Point an app at the directory and sync (adjust repo/path to your own)
argocd app create ordered-demo \
  --repo https://github.com/acme/argocd-demo.git \
  --path ordered-demo \
  --dest-server https://kubernetes.default.svc \
  --dest-namespace demo \
  --sync-option CreateNamespace=true \
  --sync-policy manual
argocd app sync ordered-demo

# (representative — trimmed)
Operation:   Sync
Phase:       Succeeded
Message:     successfully synced (all tasks run)

GROUP  KIND        NAMESPACE  NAME        STATUS     HEALTH   HOOK      MESSAGE
       Job         demo       db-migrate  Succeeded           PreSync   job.batch/db-migrate created
       ConfigMap   demo       web-config  Synced
apps   Deployment  demo       web         Synced     Healthy
       Service     demo       web         Synced     Healthy  service/web created
       Job         demo       smoke-test  Succeeded           PostSync  job.batch/smoke-test created

What just happened: read the HOOK column. db-migrate ran in the PreSync phase and shows Succeeded; the ConfigMap/Deployment/Service applied in the Sync phase (no hook); smoke-test ran in PostSync and Succeeded. The operation-level Phase: Succeeded means the whole sequence completed. Argo CD did not apply these all at once — it ran PreSync, waited, ran Sync, waited for Healthy, then ran PostSync.

Step 2 — Watch the ordering happen (the real payoff). If you sync again with the pods being watched, you see the sequence in wall-clock time:

# In one terminal, watch pods; in another, re-sync
kubectl -n demo get pods -w

# (representative — note the ORDER things appear)
NAME                    READY   STATUS      RESTARTS   AGE
db-migrate-abc12        0/1     Pending     0          0s     # PreSync: migration first
db-migrate-abc12        1/1     Running     0          2s
db-migrate-abc12        0/1     Completed   0          7s     # migration done → Sync phase begins
web-7d9c8b6f5-2xk4t     0/1     Pending     0          7s     # wave 0: app pods now
web-7d9c8b6f5-2xk4t     1/1     Running     0          12s
web-7d9c8b6f5-9mzq8     1/1     Running     0          12s    # Deployment Healthy → PostSync
smoke-test-def34        0/1     Pending     0          18s    # PostSync: smoke test last
smoke-test-def34        0/1     Completed   0          23s

What just happened: the timeline is the lesson. The db-migrate pod runs and completes before any web pod appears (PreSync gates the Sync phase). The web pods come up next and reach Ready. Only after the Deployment is Healthy does smoke-test appear (PostSync waits for health). You are watching phases and the wave-0 health gate execute in order.

Step 3 — Prove hooks re-run every sync (the idempotency lesson, live). Sync a third time and watch db-migrate run again:

argocd app sync ordered-demo
kubectl -n demo get jobs

# (representative) — db-migrate ran again this sync
NAME         STATUS     COMPLETIONS   DURATION   AGE
db-migrate   Complete   1/1           6s         8s
smoke-test   Complete   1/1           5s         2s

What just happened: the PreSync migration executed on this sync too — hooks are “run once per sync,” not “run once ever.” If our fake migration had been a non-idempotent CREATE TABLE, this second run would have failed and aborted the sync. This is exactly why real migrations must be idempotent.

Step 4 — Confirm the delete policy cleaned up. Because both hooks carry hook-delete-policy: HookSucceeded, their Jobs are deleted shortly after they succeed:

kubectl -n demo get jobs
# (a moment later — the succeeded hook Jobs are gone)
# No resources found in demo namespace.

What just happened: HookSucceeded removed each hook Job once it passed, so the namespace is not accumulating finished migration/test Jobs. Had we left the default (BeforeHookCreation), each Job would linger until the next sync replaced it. Had we used generateName with the default, they would pile up.

Teardown.

# Delete the app and everything it created (including the namespace)
argocd app delete ordered-demo --cascade

What just happened: --cascade removes the managed resources (ConfigMap, Deployment, Service) along with the Application. The hook Jobs were already gone via HookSucceeded. Your cluster is back to a clean Argo CD. You have now built and observed a real ordered deploy end to end.

---

Common mistakes and troubleshooting

Every row here is a real symptom with a real Argo CD state or message behind it. Keep it close during an incident.

Three gotchas cost the most hours, so give them extra words:

1. The migration that poisons every future sync. A PreSync migration that is not idempotent works beautifully on the deploy you tested — the first sync — and then silently becomes a landmine. The next sync (a new commit, a self-heal, anyone clicking Sync) re-runs it, it fails on “table already exists,” the PreSync phase fails, and now the whole app cannot sync at all until someone notices the migration Job’s logs. The blast radius is much larger than “one migration failed”: a non-idempotent PreSync hook can wedge your entire deployment pipeline. Treat idempotency as a hard requirement, not a nicety, and prefer a real migration tool that records applied versions.

2. The wave that waits forever. The health gate between waves is a feature — it is why your app waits for the database — but it has no built-in escape hatch. If a wave-0 resource never becomes Healthy (an image that will never pull, a probe that will never pass), Argo CD sits at wave 0 indefinitely, and every later wave and every PostSync hook never runs. From the outside this looks like “the sync is stuck,” and people go hunting in Argo CD when the cause is a Degraded pod in the current wave. The discipline: when a sync stalls, argocd app get and read the tree — the wave is almost certainly parked on one unhealthy resource, and that resource, not Argo CD, is your bug.

3. The delete policy that eats the evidence. HookFailed sounds symmetric with HookSucceeded, so people set both “to be tidy.” But HookFailed deletes a hook Job the moment it fails — including the logs you need to find out why it failed. When a migration or smoke test fails, you almost always want the failed Job and its pod to stay so you can kubectl logs it. Use HookSucceeded to clean the happy path; leave failures on the floor where you can inspect them. Reserve HookFailed for cleanup work whose failure genuinely does not matter.

---

Cheat-sheet

Bookmark this. It answers “what annotation do I need, and how do I watch it run?”

The annotations (all under metadata.annotations):

The phases, in run order:

Commands to drive and watch ordering:

Rules of thumb: migrations → PreSync + idempotent + HookSucceeded; tests → PostSync + retries; cleanup → SyncFail; “database Healthy before app” → waves, not hooks; and remember waves never cross Applications.

---

Interview and exam questions

Q: What problem do sync waves and hooks solve, and how do they differ? A: Argo CD applies an Application’s resources roughly at once, but some resources have prerequisites (a CRD before its CR, a migration before the app, a namespace before its quota). Sync waves order normal resources into numbered groups that apply in sequence, with Argo CD waiting for each group to be Healthy before the next. Hooks attach run-once work (usually a Job) to a phase of the sync — before it (PreSync), during (Sync), after all-Healthy (PostSync), or on failure (SyncFail). Waves order permanent state; hooks run transient work at sync boundaries.

Q: A resource has sync-wave: "-1". What does that mean, and what is the default wave? A: It applies in an earlier group than the default. Argo CD applies lower wave numbers first (−1 before 0 before 1), and waits for each wave to become Healthy before starting the next. The default for any un-annotated resource is wave 0. The value is a quoted string.

Q: Explain the full ordering precedence Argo CD uses in a sync. A: Four keys, coarsest to finest: phase (PreSync → Sync → PostSync, run strictly in sequence), then wave (lower first within a phase), then kind (a built-in order — Namespaces and CRDs early, custom resources late), then name (alphabetical). So the phase always dominates the wave: a PreSync hook runs before every Sync-phase resource regardless of wave numbers.

Q: Why must a PreSync migration Job be idempotent? A: Because hooks run on every sync, not just the first — a new commit, a manual sync, or a self-heal each re-runs the PreSync hook. A non-idempotent migration (CREATE TABLE with no guard) succeeds the first time and then fails on “already exists,” which fails the PreSync phase and can wedge the entire app’s syncing. Use a migration framework that records applied versions, or IF NOT EXISTS/guard queries.

Q: Name the three hook delete policies and when each deletes the hook. A: HookSucceeded deletes the hook after it succeeds; HookFailed deletes it after it fails; BeforeHookCreation (the default) deletes the previous instance right before the next sync creates a new one. For a passing one-shot you want HookSucceeded; you usually avoid HookFailed because it deletes the logs you need to debug a failure.

Q: A sync is stuck — it never leaves one wave. How do you diagnose it? A: The wave is health-gated, so it is almost certainly parked on a resource that never became Healthy. Run argocd app get, find the Degraded/Progressing child in that wave (bad image → ImagePullBackOff, failing probe, exceeded progress deadline), and fix that resource — Argo CD is correctly refusing to advance until the wave is Healthy. Every later wave and PostSync hook is blocked behind it.

Q: Your CRD and its custom resources are in the same sync and the CR apply fails with “no matches for kind.” Why, and how do you fix it? A: The custom resource was dry-run/applied before the API server finished establishing the freshly-applied CRD. Fix it by putting the CRD in an earlier sync wave than the CR, or by adding argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true to the custom resource so Argo CD does not fail the dry-run on the not-yet-known kind.

Q: What is the difference between a Sync-phase hook and a plain resource? A: Almost none in timing — an un-annotated resource is implicitly in the Sync phase. The difference is lifecycle: a hook is tracked as run-once work and governed by a delete policy, whereas a plain resource is permanent desired state that self-heals and is pruned when removed from Git. The annotation is what flips a Job between the two.

Q: Do sync waves order resources across two different Applications? A: No. Waves order resources within a single Application’s sync. Two Applications sync independently and do not honor each other’s wave numbers. To order across Applications, put waves on the child Application objects in an app-of-apps, or gate on health between stacked ApplicationSets.

Q: A Helm chart has a helm.sh/hook: pre-upgrade Job. Does it run under Argo CD? A: Yes — Argo CD renders the chart with helm template and re-expresses install/upgrade hooks as its own phases: pre-install/pre-upgrade → PreSync, post-install/post-upgrade → PostSync, and helm.sh/hook-weight behaves like a sync-wave. What does not translate: helm test, pre-delete, and *-rollback hooks, because there is no Helm test/delete/rollback lifecycle in the render model.

Q: When does a SyncFail hook fire, and when does it surprisingly not? A: It fires when the sync operation reaches a failed phase — for example a PreSync hook failed, or a resource could not apply. It does not fire on a healthy sync, and it can appear not to fire if the sync merely hung (a hook Job running forever) rather than failing, or if the operation timed out in a state Argo CD did not classify as failed. Give hooks an activeDeadlineSeconds/backoffLimit so they fail cleanly and let SyncFail trigger.

---

Key takeaways

• Two mechanisms, two questions. Sync waves order the normal resources within one sync (lowest wave first, health-gated between waves); hooks attach run-once work to a sync phase. Waves are for permanent state that needs ordering; hooks are for migrations, tests, and cleanup.
• The full order is phase → wave → kind → name, and phase always wins: a PreSync hook runs before every Sync-phase resource no matter the wave. Even with no waves, Argo CD’s built-in kind order already applies Namespaces and CRDs first.
• Waves gate on health. Argo CD applies a wave, waits for every resource in it to be Healthy, then starts the next — which is how a database (wave 0) precedes its app (wave 1). A resource that never goes Healthy stalls the sync at that wave forever.
• The four phases: PreSync (before the sync — migrations, backups; must succeed), Sync (with your resources), PostSync (after all-Healthy — smoke tests, notifications), SyncFail (on a failed operation — cleanup). Plus Skip (do not apply) and PostDelete (on app deletion).
• Hooks run on every sync, so PreSync work must be idempotent. There is no “run once ever” hook; a non-idempotent migration will fail on the second sync and can wedge the whole app. Use a migration framework or IF NOT EXISTS/guards.
• Set a delete policy or drown in Jobs. Default is BeforeHookCreation (keeps one, cleans on the next sync, not on success); use HookSucceeded to clean passing one-shots promptly; avoid HookFailed so you keep failure logs to debug.
• Helm hooks become Argo CD hooks at render time: pre-* → PreSync, post-* → PostSync, hook-weight → sync-wave, delete policies map across; test/delete/rollback hooks are not honoured.
• Waves never cross Applications, and this whole topic is cloud-neutral — identical on AKS, EKS, GKE, and kind. The only per-cloud edge is how a migration Job’s ServiceAccount authenticates to a managed database (Workload Identity / IRSA / Workload Identity), not the ordering itself.