1.11 updates (part 1) (#979)

* VAP blog minor cleanup Signed-off-by: chipzoller <[email protected]> * remove incorrect sentence re patch ops Signed-off-by: chipzoller <[email protected]> * remove inaccurate AWS statement Signed-off-by: chipzoller <[email protected]> * add wildcard match format to list Signed-off-by: chipzoller <[email protected]> * better mutate page description Signed-off-by: chipzoller <[email protected]> * add note about protected fields during mutation Signed-off-by: chipzoller <[email protected]> * cleanup docs for 1.11 Signed-off-by: chipzoller <[email protected]> * verifyImages tweaks; add blog link Signed-off-by: chipzoller <[email protected]> * extend note on Helm escaping Signed-off-by: chipzoller <[email protected]> --------- Signed-off-by: chipzoller <[email protected]> Co-authored-by: shuting <[email protected]>
kyverno · Oct 17, 2023 · 5dad325 · 5dad325
1 parent 4ee00e5
commit 5dad325
Show file tree

Hide file tree

Showing 9 changed files with 131 additions and 50 deletions.
diff --git a/content/en/blog/general/apply-vap-using-cli/index.md b/content/en/blog/general/apply-vap-using-cli/index.md
@@ -5,17 +5,18 @@ linkTitle: "Applying Validating Admission Policies using Kyverno CLI"
 author: Mariam Fahmy
 description: "Using Kyverno CLI to apply Validating Admission Policies"
 ---
-The [Kyverno Command Line Interface (CLI)](/docs/kyverno-cli/) allows applying policies outside of Kuberentes clusters and can validate and test policy behavior prior to adding them to a cluster.
+The [Kyverno Command Line Interface (CLI)](/docs/kyverno-cli/) allows applying policies outside of Kubernetes clusters and can validate and test policy behavior prior to adding them to a cluster.
 
-The two commands used for testing are “apply” and “test”:
-The apply command is used to perform a dry run on one or more policies for the given manifest(s).
-The test command is used to test a given set of resources against one or more policies to check the desired results.
+The two commands used for testing are `apply` and `test`:
 
-In this post, I will show you how the Kyverno CLI can now be used to apply/test Kubernetes Validating Admission Policies that were first introduced in 1.26.
+* The `apply` command is used to perform a dry run on one or more policies for the given manifest(s).
+* The `test` command is used to test a given set of resources against one or more policies to check the desired results defined in a special test manifest.
 
-## Applying Validating Admission Policies using kyverno apply
+In this post, I will show you how you can apply/test Kubernetes ValidatingAdmissionPolicies that were first [introduced in 1.26](https://kubernetes.io/blog/2022/12/20/validating-admission-policies-alpha/) with the enhancements to the Kyverno CLI in v1.11.
 
-In this section, you will create a validating admission policy that checks the number of deployment replicas. You will then apply this policy to two deployments, one of which violates the policy:
+## Applying ValidatingAdmissionPolicies using kyverno apply
+
+In this section, you will create a ValidatingAdmissionPolicy that checks the number of Deployment replicas. You will then apply this policy to two Deployments, one of which violates the policy:
 
 ```sh
 cat << EOF > check-deployment-replicas.yaml
@@ -37,7 +38,8 @@ spec:
 EOF
 ```
 
-The following deployment satisfies the rules declared in the above policy:
+The following deployment satisfies the rules declared in the above policy.
+
 ```sh
 cat << EOF > deployment-pass.yaml
 apiVersion: apps/v1
@@ -60,19 +62,22 @@ spec:
 EOF
 ```
 
-Let's apply the policy to the resource using `kyverno apply` as follows:
+Let's apply the policy to the resource using `kyverno apply` as follows.
+
 ```sh
 kyverno apply ./check-deployment-replicas.yaml --resource deployment-pass.yaml
 ```
 
-The output should be:
+The output should be the following.
+
 ```sh
 Applying 1 policy rule(s) to 1 resource(s)...
 
 pass: 1, fail: 0, warn: 0, error: 0, skip: 0 
 ```
 
-Let's try to create another deployment that violates the policy:
+Let's try to create another deployment that violates the policy.
+
 ```sh
 cat << EOF > deployment-fail.yaml
 apiVersion: apps/v1
@@ -95,25 +100,28 @@ spec:
 EOF
 ```
 
-Then apply the policy to the resource as follows:
+Then apply the policy to the resource as follows.
+
 ```sh
 kyverno apply ./check-deployment-replicas.yaml --resource deployment-fail.yaml 
 ```
 
-The output should be:
+The output should be as shown.
+
 ```sh
 Applying 1 policy rule(s) to 1 resource(s)...
 
 pass: 0, fail: 1, warn: 0, error: 0, skip: 0
 ```
 
-## Testing Validating Admission Policies using kyverno test
+## Testing ValidatingAdmissionPolicies using kyverno test
 
-In this section, you will create a validating admission policy that ensures no `hostPath` volumes are in use for deployments. You will then create two deployments to test them against the policy and check the desired results. 
+In this section, you will create a ValidatingAdmissionPolicy that ensures no `hostPath` volumes are in use for Deployments. You will then create two Deployments to test them against the policy and check the desired results.
 
 To proceed, you need to create a directory containing the necessary manifests. In this example, I created a directory called `test-dir`.
 
-Let's start with creating the policy:
+Let's start with creating the policy.
+
 ```sh
 cat << EOF > ./test-dir/disallow-host-path.yaml
 apiVersion: admissionregistration.k8s.io/v1alpha1
@@ -134,7 +142,8 @@ spec:
 EOF
 ```
 
-Then create the two deployments, one of which violates the policy:
+Then, create the two Deployments, one of which violates the policy.
+
 ```sh
 cat << EOF > ./test-dir/deployments.yaml
 apiVersion: apps/v1
@@ -188,7 +197,8 @@ spec:
 EOF
 ```
 
-The tests are written in a file name `kyverno-test.yaml` so you will create two tests, one for each deployment and test them against the policy.
+The tests are defined in a file named `kyverno-test.yaml` so you will create two tests, one for each Deployment and test them against the policy. Notice the use of a new field in the test manifest called `isValidatingAdmissionPolicy`.
+
 ```sh
 cat << EOF > ./test-dir/kyverno-test.yaml
 name: disallow-host-path-test
@@ -209,12 +219,15 @@ results:
     result: fail
 EOF
 ```
-Now, we’re ready to test the two deployments against validating admission policy.
+
+Now, we’re ready to test the two Deployments against a ValidatingAdmissionPolicy.
+
 ```sh
 kyverno test ./test-dir
 ```
 
-The output should be:
+The output should be as shown below.
+
 ```sh
 Executing disallow-host-path-test...
 
@@ -227,8 +240,9 @@ Executing disallow-host-path-test...
 
 Test Summary: 2 tests passed and 0 tests failed
 ```
+
 As expected, the two tests passed because the actual result of each test matches the desired result as defined in the test manifest.
 
 ## Conclusion
 
-This blog post explains how to apply validating admission policies to resources using the Kyverno CLI. With Kyverno, it's easy to apply Kubernetes Validating Admission Policies in your CI/CD pipelines and to test new Validating Admission Policies before they are deployed to your clusters.
+This blog post explains how to apply ValidatingAdmissionPolicies to resources using the Kyverno CLI. With Kyverno, it's easy to apply Kubernetes ValidatingAdmissionPolicies in your CI/CD pipelines and to test new ValidatingAdmissionPolicies before they are deployed to your clusters. This is one of many exciting features coming with Kyverno v1.11.
diff --git a/content/en/docs/Installation/customization.md b/content/en/docs/Installation/customization.md
@@ -311,10 +311,11 @@ The following flags can be used to control the advanced behavior of the various
 52. `tracingCreds` (ABCR): set to the CA secret containing the certificate which is used by the Opentelemetry Tracing Client. If empty string is set, an insecure connection will be used.
 53. `tracingPort` (ABCR): tracing receiver port. Default is `"4317"`.
 54. `transportCreds` (ABCR): set to the CA secret containing the certificate used by the OpenTelemetry metrics client. Empty string means an insecure connection will be used. Default is `""`.
-55. `v` (ABCR): sets the verbosity level of Kyverno log output. Takes an integer from 1 to 6 with 6 being the most verbose. Level 4 shows variable substitution messages. Default is `2`.
-56. `vmodule` (ABCR): comma-separated list of pattern=N settings for file-filtered logging.
-57. `webhookRegistrationTimeout` (A): specifies the length of time Kyverno will try to register webhooks with the API server. Defaults to `120s`.
-58. `webhookTimeout` (A): specifies the timeout for webhooks, in seconds. After the timeout passes, the webhook call will be ignored or the API call will fail based on the failure policy. The timeout value must be an integer number between 1 and 30 (seconds). Defaults is `10`.
+55. `ttlReconciliationInterval` (C): defines the interval the cleanup controller should perform reconciliation of resources labeled with the cleanup TTL label. Default is `1m`. See the cleanup documentation [here](/docs/writing-policies/cleanup/#cleanup-label) for details.
+56. `v` (ABCR): sets the verbosity level of Kyverno log output. Takes an integer from 1 to 6 with 6 being the most verbose. Level 4 shows variable substitution messages. Default is `2`.
+57. `vmodule` (ABCR): comma-separated list of pattern=N settings for file-filtered logging.
+58. `webhookRegistrationTimeout` (A): specifies the length of time Kyverno will try to register webhooks with the API server. Defaults to `120s`.
+59. `webhookTimeout` (A): specifies the timeout for webhooks, in seconds. After the timeout passes, the webhook call will be ignored or the API call will fail based on the failure policy. The timeout value must be an integer number between 1 and 30 (seconds). Defaults is `10`.
 
 ### Policy Report access
 

diff --git a/content/en/docs/Writing policies/cleanup.md b/content/en/docs/Writing policies/cleanup.md
@@ -1,22 +1,24 @@
 ---
 title: Cleanup Rules
 description: >
-  Remove Kubernetes resources. 
+  Remove Kubernetes resources.
 weight: 70
 ---
 
 {{% alert title="Warning" color="warning" %}}
-Cleanup policies are an **alpha** feature. It is not ready for production usage and there may be breaking changes. Normal semantic versioning and compatibility rules will not apply.
+Cleanup policies are a **beta** feature. It is not ready for production usage and there may be breaking changes. Normal semantic versioning and compatibility rules will not apply.
 {{% /alert %}}
 
-Kyverno has the ability to cleanup (i.e., delete) existing resources in a cluster defined in a new policy called a `CleanupPolicy`. Cleanup policies come in both cluster-scoped and Namespaced flavors; a `ClusterCleanupPolicy` being cluster scoped and a `CleanupPolicy` being Namespaced. A cleanup policy uses the familiar `match`/`exclude` block to select and exclude resources which are subjected to the cleanup process. A `conditions{}` block (optional) uses common expressions similar to those found in [preconditions](/docs/writing-policies/preconditions/) and [deny rules](/docs/writing-policies/validate/#deny-rules) to query the contents of the selected resources in order to refine the selection process. And, lastly, a `schedule` field defines, in cron format, when the rule should run.
+Kyverno has the ability to cleanup (i.e., delete) existing resources in a cluster in two different ways. The first way is via a declarative policy definition in either a `CleanupPolicy` or `ClusterCleanupPolicy`. See the section on [cleanup policies](#cleanup-policy) below for more details. The second way is via a reserved time-to-live (TTL) label added to a resource. See the [cleanup label](#cleanup-label) section for further details.
+
+## Cleanup Policy
+
+Similar to other policies which can validate, mutate, generate, or verify images in resources, Kyverno can cleanup resources by defining a new policy type called a `CleanupPolicy`. Cleanup policies come in both cluster-scoped and Namespaced flavors; a `ClusterCleanupPolicy` being cluster scoped and a `CleanupPolicy` being Namespaced. A cleanup policy uses the familiar `match`/`exclude` block to select and exclude resources which are subjected to the cleanup process. A `conditions{}` block (optional) uses common expressions similar to those found in [preconditions](/docs/writing-policies/preconditions/) and [deny rules](/docs/writing-policies/validate/#deny-rules) to query the contents of the selected resources in order to refine the selection process. [Context variables](/docs/writing-policies/external-data-sources/) (optional) can be used to fetch data from other resources to factor into the cleanup process. And, lastly, a `schedule` field defines, in cron format, when the rule should run.
 
 {{% alert title="Note" color="info" %}}
 Since cleanup policies always operate against existing resources in a cluster, policies created with `subjects`, `Roles`, or `ClusterRoles` in the `match`/`exclude` block are not allowed since this information is only known at admission time.
 {{% /alert %}}
 
-The cleanup controller runs decoupled from Kyverno in a separate Deployment. Cleanup is executed by a CronJob which is automatically created and managed by the cleanup controller. Each cleanup policy maps to one CronJob. When the scheduled time occurs, the CronJob calls to the cleanup controller to execute the cleanup process defined in the policy. As cleanup policies are either updated or removed, the CronJobs are updated accordingly.
-
 An example ClusterCleanupPolicy is shown below. This cleanup policy removes Deployments which have the label `canremove: "true"` if they have less than two replicas on a schedule of every 5 minutes.
 
 ```yaml
@@ -45,6 +47,56 @@ Values from resources to be evaluated during a policy may be referenced with `ta
 
 Because Kyverno follows the principal of least privilege, depending on the resources you wish to remove it may be necessary to grant additional permissions to the cleanup controller. Kyverno will assist in informing you if additional permissions are required by validating them at the time a new cleanup policy is installed. See the [Customizing Permissions](/docs/installation/customization/#customizing-permissions) section for more details.
 
-{{% alert title="Warning" color="warning" %}}
-Be mindful of the validate policies in `Enforce` mode in your cluster as the CronJobs and their spawned Jobs/Pods may be subjected to and potentially blocked. You may wish to exclude based on the label `app.kubernetes.io/managed-by`.
-{{% /alert %}}
+An example ClusterRole which allows Kyverno to cleanup Pods is shown below. This may need to be customized based on the values used to deploy Kyverno.
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  labels:
+    app.kubernetes.io/component: cleanup-controller
+    app.kubernetes.io/instance: kyverno
+    app.kubernetes.io/part-of: kyverno
+  name: kyverno:cleanup-pods
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  verbs:
+  - get
+  - watch
+  - list
+  - delete
+```
+
+## Cleanup Label
+
+In addition to policies which can declaratively define what resources to remove and when to remove them, the second option for cleanup involves assignment of a reserved label called `cleanup.kyverno.io/ttl` to the exact resource(s) which should be removed. The value of this label can be one of two supported formats. Any unrecognized formats will trigger a warning.
+
+* An absolute time specified in ISO 8601 format (ex., `2023-10-04T003000Z` or `2023-10-04`)
+* A remaining time calculated from when the label was observed (ex., `5m`, `4h`, or `1d`)
+
+This label can be assigned to any resource and so long as Kyverno has the needed permissions to delete the resource (see above section for an example), it will be removed at the designated time.
+
+For example, creation of this Pod will cause Kyverno to clean it up after two minutes and without the presence of a cleanup policy.
+
+```yaml
+apiVersion: v1
+kind: Pod
+metadata:
+  labels:
+    cleanup.kyverno.io/ttl: 2m
+  name: foo
+spec:
+  containers:
+  - args:
+    - sleep
+    - 1d
+    image: busybox:1.35
+    name: foo
+```
+
+Although labeled resources are watched by Kyverno, the cleanup interval (the time resolution at which any cleanup can be performed) is controlled by a flag passed to the cleanup controller called `ttlReconciliationInterval`. This value is set to `1m` by default and can be changed if a longer resolution is required.
+
+Because this is a label, there is opportunity to chain other Kyverno functionality around it. For example, it is possible to use a Kyverno mutate rule to assign this label to matching resources. A validate rule could be written prohibiting, for example, users from the `infra-ops` group from assigning the label to resources in certain Namespaces. Or, Kyverno could generate a new resource with this label as part of the resource definition.
diff --git a/content/en/docs/Writing policies/match-exclude.md b/content/en/docs/Writing policies/match-exclude.md
@@ -43,6 +43,7 @@ To resolve kind naming conflicts, specify the API group and version. For example
 Wildcards are supported with the following formats:
 
 * `Group/*/Kind`
+* `Group/*/*`
 * `*/Kind`
 * `*`
 

diff --git a/content/en/docs/Writing policies/mutate.md b/content/en/docs/Writing policies/mutate.md
@@ -1,14 +1,18 @@
 ---
 title: Mutate Rules
 description: >
-  Modify resource configurations.
+  Modify resource configurations during admission or retroactively against existing resources.
 weight: 40
 ---
 
 A `mutate` rule can be used to modify matching resources and is written as either a RFC 6902 JSON Patch or a strategic merge patch.
 
 By using a patch in the [JSONPatch - RFC 6902](http://jsonpatch.com/) format, you can make precise changes to the resource being created. A strategic merge patch is useful for controlling merge behaviors on elements with lists. Regardless of the method, a `mutate` rule is used when an object needs to be modified in a given way.
 
+{{% alert title="Note" color="info" %}}
+Kubernetes disallows changes to certain fields in resources including `name`, `namespace`, `uid`, `kind`, and `apiVersion`, therefore you cannot use Kyverno policy to effect any of these fields either during admission or once a resource has been persisted.
+{{% /alert %}}
+
 Resource mutation occurs before validation, so the validation rules should not contradict the changes performed by the mutation section. To mutate existing resources in addition to those subject to AdmissionReview requests, use [mutateExisting](#mutate-existing-resources) policies.
 
 This policy sets the `imagePullPolicy` to `IfNotPresent` if the image tag is `latest`:
@@ -44,8 +48,6 @@ A [JSON Patch](http://jsonpatch.com/), implemented as a mutation method called `
 * `replace`
 * `remove`
 
-With Kyverno, the `add` and `replace` have the same behavior (i.e., both operations will add or replace the target element).
-
 The `patchesJson6902` method can be useful when a specific mutation is needed which cannot be performed by `patchesStrategicMerge`. For example, when needing to mutate a specific object within an array, the index can be specified as part of a `patchesJson6902` mutation rule.
 
 One distinction between this and other mutation methods is that `patchesJson6902` does not support the use of conditional anchors. Use [preconditions](/docs/writing-policies/preconditions/) instead. Also, mutations using `patchesJson6902` to Pods directly are not converted to higher-level controllers such as Deployments and StatefulSets through the use of the [auto-gen feature](/docs/writing-policies/autogen/). Therefore, when writing such mutation rules for Pods, it may be necessary to create multiple rules to cover all relevant Pod controllers.

diff --git a/content/en/docs/Writing policies/variables.md b/content/en/docs/Writing policies/variables.md
@@ -182,6 +182,20 @@ Because Helm executes its templating routine prior to Kyverno, a Kyverno policy
 {{`{{ request.userInfo.username }}`}}
 ```
 
+For Kyverno variables which use more complex JMESPath expressions including [existence checks](/docs/writing-policies/jmespath/#non-existence-checks), elements such as empty objects or arrays may also need to be escaped even within the overall expression. For example, the value of the below `value` field
+
+```
+value: "{{ element.securityContext.capabilities.drop[].to_upper(@) || `[]` }}"
+```
+
+would need to become
+
+```
+value: {{ `"{{ element.securityContext.capabilities.drop[].to_upper(@) || `}}`[]`{{` }}"` }}
+```
+
+in order to render properly.
+
 ## Variables from admission review requests
 
 Kyverno operates as a webhook inside Kubernetes. Whenever a new request is made to the Kubernetes API server, for example to create a Pod, the API server sends this information to the webhooks registered to listen to the creation of Pod resources. This incoming data to a webhook is passed as a [`AdmissionReview`](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#webhook-request-and-response) object. There are four commonly used data properties available in any AdmissionReview request: