Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added documentation for DeletionPropagationPolicy for cleanupPolicy a… #1426

Merged
128 changes: 103 additions & 25 deletions content/en/docs/writing-policies/cleanup.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,17 +25,17 @@ metadata:
spec:
match:
any:
- resources:
kinds:
- Deployment
selector:
matchLabels:
canremove: "true"
- resources:
kinds:
- Deployment
selector:
matchLabels:
canremove: "true"
conditions:
any:
- key: "{{ target.spec.replicas }}"
operator: LessThan
value: 2
- key: "{{ target.spec.replicas }}"
operator: LessThan
value: 2
schedule: "*/5 * * * *"
```

Expand All @@ -55,23 +55,23 @@ metadata:
app.kubernetes.io/part-of: kyverno
name: kyverno:cleanup-pods
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- watch
- list
- delete
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- watch
- list
- delete
```

## Cleanup Label

In addition to policies which can declaratively define what resources to remove and when to remove them, the second option for cleanup involves assignment of a reserved label called `cleanup.kyverno.io/ttl` to the exact resource(s) which should be removed. The value of this label can be one of two supported formats. Any unrecognized formats will trigger a warning.

* An absolute time specified in ISO 8601 format (ex., `2023-10-04T003000Z` or `2023-10-04`)
* A remaining time calculated from when the label was observed (ex., `5m`, `4h`, or `1d`)
- An absolute time specified in ISO 8601 format (ex., `2023-10-04T003000Z` or `2023-10-04`)
- A remaining time calculated from when the label was observed (ex., `5m`, `4h`, or `1d`)

This label can be assigned to any resource and so long as Kyverno has the needed permissions to delete the resource (see above section for an example), it will be removed at the designated time.

Expand All @@ -86,13 +86,91 @@ metadata:
name: foo
spec:
containers:
- args:
- sleep
- 1d
image: busybox:1.35
name: foo
- args:
- sleep
- 1d
image: busybox:1.35
name: foo
```

Although labeled resources are watched by Kyverno, the cleanup interval (the time resolution at which any cleanup can be performed) is controlled by a flag passed to the cleanup controller called `ttlReconciliationInterval`. This value is set to `1m` by default and can be changed if a longer resolution is required.

Because this is a label, there is opportunity to chain other Kyverno functionality around it. For example, it is possible to use a Kyverno mutate rule to assign this label to matching resources. A validate rule could be written prohibiting, for example, users from the `infra-ops` group from assigning the label to resources in certain Namespaces. Or, Kyverno could generate a new resource with this label as part of the resource definition.

## DeletionPropagationPolicy

The `deletionPropagationPolicy` field is an optional setting available in both CleanupPolicy and TTL-based cleanup configurations. It determines how Kubernetes handles the deletion of dependent resources when the primary resource is deleted.

Supported values:

- **Foreground**: Ensures dependent resources are deleted before the primary resource is removed.
- **Background**: Deletes the primary resource first, while dependents are removed asynchronously.
- **Orphan**: Deletes the primary resource but leaves its dependents untouched.

{{% alert title="Note" color="info" %}}
If `deletionPropagationPolicy` is not set, Kyverno defers to the Kubernetes API server's default behavior, which typically handles dependents based on cluster settings.
{{% /alert %}}

### Cleanup Policy Example with deletionPropagationPolicy

A ClusterCleanupPolicy can include `deletionPropagationPolicy` to control the cleanup of dependents. Here's an example:

```yaml
apiVersion: kyverno.io/v2
kind: ClusterCleanupPolicy
metadata:
name: cleandeploy
spec:
match:
any:
- resources:
kinds:
- Deployment
selector:
matchLabels:
canremove: "true"
conditions:
any:
- key: "{{ target.spec.replicas }}"
operator: LessThan
value: 2
schedule: "*/5 * * * *"
# use Foreground deletion propagation policy
deletionPropagationPolicy: Foreground
```

This policy schedules the deletion of Deployments labeled `canremove: "true"` with fewer than two replicas every 5 minutes, using the `Foreground` deletion propagation policy, ensuring dependent resources are deleted before the Deployment itself.

### TTL-Based Cleanup Example with deletionPropagationPolicy

Resources with a `cleanup.kyverno.io/ttl` label can also specify a deletion propagation policy to manage dependent resources:

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-server
labels:
cleanup.kyverno.io/ttl: 2m
annotations:
# use Foreground deletion propagation policy
cleanup.kyverno.io/propagation-policy: Foreground
spec:
replicas: 2
selector:
matchLabels:
app: nginx-server
template:
metadata:
labels:
app: nginx-server
spec:
containers:
- name: nginx-server
image: nginx
```

In this example:
ShivamJha2436 marked this conversation as resolved.
Show resolved Hide resolved

- The TTL label specifies that the resource will be deleted 2 minutes after creation.
- The deletion propagation policy `Foreground` ensures that any dependent resources in the cluster are deleted before the resource itself.
Loading