The Silent Killer in Your ArgoCD Configuration: Why preserveResourcesOnDeletion in syncOptions Does Nothing

Huub Daems

You set preserveResourcesOnDeletion=true in your ApplicationSet’s syncOptions. You deployed it. You deleted the ApplicationSet. Everything was deleted anyway.

No error. No warning. No log message. Just silence — and then gone.

We recently helped a customer recover from exactly this scenario. A hub-and-spoke ArgoCD architecture lost all deployed resources across two clusters because preserveResourcesOnDeletion was placed in the wrong field. This post explains what happened, why it’s so dangerous, and how to get it right.

The Setup: Hub-and-Spoke ArgoCD

The customer ran a common pattern: one management cluster hosting a parent ArgoCD instance and several child ArgoCD instances (a-gitops, b-gitops, c-gitops, d-gitops). The parent deploys the children; the children deploy workloads to the management cluster and a separate workload cluster.

Management Cluster (Cluster-I)
├── Parent ArgoCD (openshift-gitops namespace)
│   ├── Deploys child ArgoCD instances
│   └── Deploys cluster config via ApplicationSets
├── Child ArgoCD: a-gitops
├── Child ArgoCD: b-gitops
├── Child ArgoCD: c-gitops
└── Child ArgoCD: d-gitops

Workload Cluster (Cluster-II)
└── Deployed resources only (no GitOps instances)

This architecture is elegant — until you need to remove it.

The Incident: Two-Level Cascading Delete

The customer wanted to migrate their GitOps infrastructure to a new cluster. Step one: uninstall the GitOps operator on the management cluster.

What happened next was a two-level cascading delete:

  1. Level 1: Uninstalling GitOps on Cluster-I deleted the parent ArgoCD. The parent’s child ArgoCD instances (themselves Applications of the parent) were cascade-deleted via Kubernetes ownerReferences.
  2. Level 2: Each child ArgoCD deletion triggered resources-finalizer.argocd.argoproj.io on every Application managed by that child. The finalizer tells ArgoCD: “before you delete me, delete all my deployed resources.” And ArgoCD obeys — on all target clusters.

Result: the customer was locked out of both clusters. Namespaces, OAuth configurations, RBAC bindings — everything was gone.

The Misconfiguration: preserveResourcesOnDeletion in syncOptions

The customer had tried to protect themselves. They set preserveResourcesOnDeletion=true in their ApplicationSet syncOptions:

spec:
  template:
    spec:
      syncPolicy:
        syncOptions:
          - ApplyOutOfSyncOnly=true
          - preserveResourcesOnDeletion=true   # <-- THIS DOES NOTHING

This looks reasonable. preserveResourcesOnDeletion sounds like a sync option. It’s even listed that way in some blog posts and community discussions. But ArgoCD does not recognize preserveResourcesOnDeletion as a valid syncOption. It is silently ignored.

Why Is It Silently Ignored?

ArgoCD’s Sync Options documentation lists every valid syncOption: Prune, Validate, SkipDryRunOnMissingResource, Delete, ApplyOutOfSyncOnly, PrunePropagationPolicy, PruneLast, Replace, Force, ServerSideApply, ClientSideApplyMigration, FailOnSharedResource, RespectIgnoreDifferences, CreateNamespace.

preserveResourcesOnDeletion is not on that list.

It is documented in exactly one place: the ApplicationSet Application-Deletion page, which describes it as a top-level syncPolicy field — not a syncOption.

When you put an unrecognized string in syncOptions, ArgoCD doesn’t throw an error. It just ignores it. Your ApplicationSet looks protected. It isn’t.

The Correct Configuration

The only valid placement for preserveResourcesOnDeletion is at the ApplicationSet level, as a top-level syncPolicy field:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: my-applicationset
spec:
  syncPolicy:
    preserveResourcesOnDeletion: true    # CORRECT: ApplicationSet-level syncPolicy field
  template:
    spec:
      syncPolicy:
        syncOptions:
          - ApplyOutOfSyncOnly=true

This tells the ApplicationSet controller: “when you generate Applications from this ApplicationSet, do NOT add the resources-finalizer.argocd.argoproj.io finalizer to them.” Without the finalizer, deleting the Application (or the ApplicationSet) does not cascade into deployed resources.

Three Steps to Protect Your Resources

Setting preserveResourcesOnDeletion: true is necessary but not sufficient. Here’s the complete protection plan:

1. Set it on EVERY ApplicationSet

In a hub-and-spoke architecture, preserveResourcesOnDeletion must be set on every ApplicationSet across all ArgoCD instances — parent and children. If even one ApplicationSet is missing it, the cascade fires through that unprotected path.

2. Remove existing finalizers

preserveResourcesOnDeletion: true only prevents the finalizer from being added to new Applications. It does NOT retroactively remove finalizers from Applications that already have them. If your Applications were created before you set preserveResourcesOnDeletion, they still carry the finalizer. You must remove it manually:

# Find Applications that still have the finalizer
oc get applications -n openshift-gitops -o json | \
  jq -r '.items[] | select(.metadata.finalizers[]? | contains("resources-finalizer.argocd.argoproj.io")) | .metadata.name'

# Remove the finalizer
oc patch application/<APP_NAME> -n openshift-gitops --type=json \
  -p '[{"op": "remove", "path": "/metadata/finalizers"}]'

3. Delete ApplicationSets with –cascade=orphan

Even with preserveResourcesOnDeletion: true, deleting an ApplicationSet triggers Kubernetes garbage collection via ownerReferences, which cascade-deletes the generated Application objects. Use --cascade=orphan to prevent this:

oc delete applicationset <NAME> -n openshift-gitops --cascade=orphan

How to Audit Your Configuration

Check for the misconfiguration we found:

# Find ApplicationSets with preserveResourcesOnDeletion in syncOptions (silently ignored!)
oc get applicationsets -A -o json | \
  jq -r '.items[] | select(.spec.template.spec.syncPolicy.syncOptions[]? | test("preserveResourcesOnDeletion")) | .metadata.name'

Check for Applications that still carry the dangerous finalizer:

oc get applications -A -o json | \
  jq -r '.items[] | select(.metadata.finalizers[]? | contains("resources-finalizer.argocd.argoproj.io")) | "\(.metadata.namespace)/\(.metadata.name)"'

Check which ApplicationSets are properly protected:

# Protected
oc get applicationsets -A -o json | \
  jq -r '.items[] | select(.spec.syncPolicy.preserveResourcesOnDeletion == true) | "\(.metadata.namespace)/\(.metadata.name)"'

# Unprotected (vulnerable to cascading deletion)
oc get applicationsets -A -o json | \
  jq -r '.items[] | select(.spec.syncPolicy.preserveResourcesOnDeletion != true) | "\(.metadata.namespace)/\(.metadata.name)"'

Lessons Learned

  1. Silent failures are the most dangerous failures. ArgoCD’s decision to silently ignore unrecognized syncOptions means you can deploy what looks like a protective configuration and never know it’s not working — until it’s too late.
  2. “Two levels” of cascade is a pattern unique to hub-and-spoke ArgoCD. The standard ArgoCD documentation discusses Application deletion cascading (Level 2). But when a parent ArgoCD manages child ArgoCDs, there’s an additional cascade level (Level 1) that amplifies the blast radius across all clusters.
  3. preserveResourcesOnDeletion is not a sync option. It’s a syncPolicy field. The difference matters. A syncOption goes on generated Applications. A syncPolicy field controls how the ApplicationSet controller generates those Applications. preserveResourcesOnDeletion controls the generator behavior — “don’t add the finalizer” — not the Application behavior.
  4. Always verify your protection works. After setting preserveResourcesOnDeletion: true, check that newly generated Applications do NOT have resources-finalizer.argocd.argoproj.io in their metadata.finalizers. If they do, the protection isn’t working.

References