From 3dc4de59ba6b09de931cd00cdf256c8d70f7076a Mon Sep 17 00:00:00 2001 From: Christoph Hartmann Date: Fri, 26 May 2023 17:03:22 +0200 Subject: [PATCH] =?UTF-8?q?=F0=9F=A7=B9=20format=20docs=20(#810)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * ⭐️ add make commands to setup prettier * 🧹 format docs --- .gitignore | 2 + .prettierignore | 6 + .prettierrc.json | 1 + README.md | 4 +- RELEASE.md | 1 + docs/operator-upgrades.md | 59 +++-- docs/rbac.md | 258 ++++++++++---------- docs/user-manual.md | 478 ++++++++++++++++++++++---------------- 8 files changed, 457 insertions(+), 352 deletions(-) create mode 100644 .prettierignore create mode 100644 .prettierrc.json diff --git a/.gitignore b/.gitignore index 6e9eaa7e6..9d6842844 100644 --- a/.gitignore +++ b/.gitignore @@ -47,3 +47,5 @@ tests/integration/_output /cnquery /cnspec + +package.json \ No newline at end of file diff --git a/.prettierignore b/.prettierignore new file mode 100644 index 000000000..d7317b25e --- /dev/null +++ b/.prettierignore @@ -0,0 +1,6 @@ +.github/** +.vscode/** +config/** +tests/** +charts/** +controllers/** \ No newline at end of file diff --git a/.prettierrc.json b/.prettierrc.json new file mode 100644 index 000000000..0967ef424 --- /dev/null +++ b/.prettierrc.json @@ -0,0 +1 @@ +{} diff --git a/README.md b/README.md index a5608023d..e256bc183 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ The Mondoo Operator provides the following features: It is backed by Mondoo's powerful policy-as-code engine [cnspec](https://mondoo.com/docs/cnspec/cnspec-about/) and [MQL](https://mondoo.com/docs/mql/resources/). Mondoo ships out-of-the-box security policies for: - CIS Kubernetes Benchmarks -- CIS AKS/EKS/GKE/OpenShift Benchmarks +- CIS AKS/EKS/GKE/OpenShift Benchmarks - NSA/CISA Kubernetes Hardening Guide - Kubernetes Cluster and Workload Security - Kubernetes Best Practices @@ -66,4 +66,4 @@ Join the [Mondoo Community GitHub Discussions](https://github.com/orgs/mondoohq/ ## License -[Mozilla Public License v2.0](https://github.com/mondoohq/mondoo-operator/blob/main/LICENSE) \ No newline at end of file +[Mozilla Public License v2.0](https://github.com/mondoohq/mondoo-operator/blob/main/LICENSE) diff --git a/RELEASE.md b/RELEASE.md index fa4ad43db..174ebaddd 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -3,6 +3,7 @@ This document describes the release process for the operator ## Versioning + Always consider what is the suitable version number to be released based on the [Operator upgrade manual](docs/operator-upgrades.md). ## Release script diff --git a/docs/operator-upgrades.md b/docs/operator-upgrades.md index 3c3104366..1d9db1a64 100644 --- a/docs/operator-upgrades.md +++ b/docs/operator-upgrades.md @@ -1,12 +1,15 @@ # Upgrading the Mondoo Operator + The Mondoo Operator version numbers are expressed as `x.y.z`, where `x` is the major version, `y` is the minor version, and `z` is the patch version, following [Semantic Versioning](https://semver.org/) standards. Our release approach ensures there are no breaking changes between two adjacent minor versions. For example, you can upgrade the Mondoo Operator from `v0.2.15` to `v0.3.0` without any manual actions. The Mondoo Operator automatically executes any required migration and/or cleanup steps. You can skip Mondoo Operator patch and minor versions. ## Version 1.0 and later + As of version 1.0, the Mondoo Operator can be upgraded from major version to major version as long as each major version is visited during the upgrade. For example, if you are on version 1.3.2 and wanted to upgrade to version 3.0.5, you should first upgrade to a 2.y.z release to ensure any migrations that need to take place from 1.x to 2.x are completed before upgrading to 3.x. Jumping from any 1.x to any 2.x is a safe operation. ## Pre 1.0 + For pre-1.0 releases, we don't recommend skipping minor versions. For example, if you upgrade from `v0.2.0` directly to `v0.4.0`, the Mondoo Operator may not behave as expected, and you may leave behind unused resources in the cluster. Skipping a minor version may require manual actions to ensure the Mondoo Operator is fully functional. **NOTE: all upgrade text below is written against 1.0 behavior. If using pre-1.0, then all mentions that follow of checking major versions should be read as checking against minor versions.** @@ -14,67 +17,81 @@ For pre-1.0 releases, we don't recommend skipping minor versions. For example, i **WARNING: Never try to upgrade the Mondoo Operator by simply changing the tag for the Mondoo Operator container image.** ## Recommended Mondoo Operator upgrade process + Follow these steps for a smooth Mondoo Operator upgrade: + 1. Verify the Mondoo Operator version currently running in the cluster: - ```bash - kubectl get deployment -n mondoo-operator mondoo-operator-controller-manager -o jsonpath='{.spec.template.spec.containers[0].image}' - ``` + ```bash + kubectl get deployment -n mondoo-operator mondoo-operator-controller-manager -o jsonpath='{.spec.template.spec.containers[0].image}' + ``` 2. Check the latest version of the Mondoo Operator on our [Releases](https://github.com/mondoohq/mondoo-operator/releases/latest) GitHub page. Based on the version difference between your Mondo Operator and the latest release, follow the steps below. -### If your current Mondoo Operator is no more than one minor version behind +### If your current Mondoo Operator is no more than one minor version behind + If there is **not** more than one major version difference between the installed Mondoo Operator and the latest release, apply the latest manifest to the cluster: + ```bash kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml ``` ### If your current Mondoo Operator is more than one major version behind + If there **is** more than one major version difference between the installed Mondoo Operator and the latest release, you must apply the manifest files for each major version between the two versions. For example, if the version installed is `v1.2.0` and the latest version is `v3.4.3`, you must install something from the `v2.x` releases. Follow these steps: 1. Apply the manifest for `v2.0.0` (the version you skipped): - ```bash - kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/v2.0.0/download/mondoo-operator-manifests.yaml - ``` + ```bash + kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/v2.0.0/download/mondoo-operator-manifests.yaml + ``` 2. Wait until the new version of the Mondoo Operator is running and verify there are no errors in the operator log: - ```bash - kubectl logs -n mondoo-operator deployment/mondoo-operator-controller-manager - ``` - It's essential to check the logs and wait for the new version of the operator to run; directly upgrading to the next version can result in skipped internal upgrade procedures and unexpected behavior. - Check the `Status.ReconciledByOperatorVersion` field of your `MondooAuditConfig` to be sure the new operator version reconciled every object: - ```bash - kubectl get mondooauditconfigs.k8s.mondoo.com -o jsonpath='{range .items[*]}{.status.reconciledByOperatorVersion}{"\n"}{end}' -A | uniq - ``` - The version of your running Mondoo Operator and the version in the `Status` field have to be the same before you can proceed with the next version update. + ```bash + kubectl logs -n mondoo-operator deployment/mondoo-operator-controller-manager + ``` + + It's essential to check the logs and wait for the new version of the operator to run; directly upgrading to the next version can result in skipped internal upgrade procedures and unexpected behavior. + + Check the `Status.ReconciledByOperatorVersion` field of your `MondooAuditConfig` to be sure the new operator version reconciled every object: + + ```bash + kubectl get mondooauditconfigs.k8s.mondoo.com -o jsonpath='{range .items[*]}{.status.reconciledByOperatorVersion}{"\n"}{end}' -A | uniq + ``` + + The version of your running Mondoo Operator and the version in the `Status` field have to be the same before you can proceed with the next version update. 3. Apply the manifest for `v3.4.3` (the latest version): - ```bash - kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml - ``` -Adjust the steps above to fit your current situation. There may be multiple major release versions between your installed version and the latest release. You must install each major version independently, wait between each update to verify that the version installed properly and the log is error-free. + `bash kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml ` + Adjust the steps above to fit your current situation. There may be multiple major release versions between your installed version and the latest release. You must install each major version independently, wait between each update to verify that the version installed properly and the log is error-free. ## Upgrading to Mondoo Operator v0.8.0 + In case you are running a Mondoo Operator with a version older than v0.8.0 in your cluster, it is required to perform extra steps before upgrading. ### Helm and kubectl installations + For Helm and kubectl installations before applying the `v0.8.0` manifests run: + ```bash kubectl delete -n mondoo-operator deployments.apps mondoo-operator-controller-manager ``` ### OLM installations + For OLM installations first list the subscriptions: + ```bash kubectl get subscription -n mondoo-operator ``` Delete the Mondoo Operator subscription: + ```bash -kubectl delete sub -n mondoo-operator mondoo-operator-v0-7-1-sub +kubectl delete sub -n mondoo-operator mondoo-operator-v0-7-1-sub ``` Delete the Mondoo Operator cluster service version: + ```bash kubectl delete csv -n mondoo-operator mondoo-operator.v0.7.1 ``` diff --git a/docs/rbac.md b/docs/rbac.md index d03061de5..4a20c8291 100644 --- a/docs/rbac.md +++ b/docs/rbac.md @@ -12,121 +12,121 @@ kind: ClusterRole metadata: name: manager-role rules: -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - list - - watch -- apiGroups: - - admissionregistration.k8s.io - resources: - - validatingwebhookconfigurations - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - daemonsets - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - apps - resources: - - deployments - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - cert-manager.io - resources: - - certificates - - issuers - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - configmaps - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - "" - resources: - - pods - verbs: - - get - - list - - watch -- apiGroups: - - "" - resources: - - services - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - k8s.mondoo.com - resources: - - mondooauditconfigs - verbs: - - create - - delete - - get - - list - - patch - - update - - watch -- apiGroups: - - k8s.mondoo.com - resources: - - mondooauditconfigs/finalizers - verbs: - - update -- apiGroups: - - k8s.mondoo.com - resources: - - mondooauditconfigs/status - verbs: - - get - - patch - - update + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - list + - watch + - apiGroups: + - admissionregistration.k8s.io + resources: + - validatingwebhookconfigurations + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - apps + resources: + - daemonsets + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - apps + resources: + - deployments + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - cert-manager.io + resources: + - certificates + - issuers + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - "" + resources: + - pods + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - services + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - k8s.mondoo.com + resources: + - mondooauditconfigs + verbs: + - create + - delete + - get + - list + - patch + - update + - watch + - apiGroups: + - k8s.mondoo.com + resources: + - mondooauditconfigs/finalizers + verbs: + - update + - apiGroups: + - k8s.mondoo.com + resources: + - mondooauditconfigs/status + verbs: + - get + - patch + - update ``` ## RBAC rules for Mondoo Workload Scanning @@ -135,22 +135,22 @@ To scan the Kubernetes resources, the mondoo-client needs access to the Kubernet As Mondoo does not modify any Objects in the Kubernetes API, but just reads them it simply requires the `get`, `list`, and `watch` actions. - ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: workload rules: -- apiGroups: - - '*' - resources: - - '*' - verbs: - - get - - watch - - list + - apiGroups: + - "*" + resources: + - "*" + verbs: + - get + - watch + - list ``` + ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -161,10 +161,11 @@ roleRef: kind: ClusterRole name: workload subjects: -- kind: ServiceAccount - name: workload - namespace: system + - kind: ServiceAccount + name: workload + namespace: system ``` + ```yaml apiVersion: v1 kind: ServiceAccount @@ -201,9 +202,6 @@ spec: To scan the Kubernetes nodes, Mondoo does not does not require access to the Kubernetes API server, thus a default service account with no permissions should suffice. - > The `ServiceAccount` used by the node-scanner `Pod`s can be specified in the `MondooAuditConfig` object. - - > See [Using Authorization Plugins](https://kubernetes.io/docs/reference/access-authn-authz/authorization/) for further usage information on RBAC components. diff --git a/docs/user-manual.md b/docs/user-manual.md index 966081778..d3f88b5a5 100644 --- a/docs/user-manual.md +++ b/docs/user-manual.md @@ -1,10 +1,13 @@ # User manual + This user manual describes how to install and use the Mondoo Operator. ## Mondoo Operator Installation + Install the Mondoo Operator using kubectl, Helm, or Operator Lifecycle Manager. ### Installing with kubectl + Follow this step to set up the Mondoo Operator using kubectl and a manifest file. Precondition: kubectl with cluster admin access @@ -14,7 +17,7 @@ To install with kubectl, apply the operator manifests: ```bash kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml ``` - + or ```bash @@ -23,98 +26,110 @@ kubectl apply -f mondoo-operator-manifests.yaml ``` ### Installing with Helm + Follow these steps to set up the Mondoo Operator using [Helm](https://helm.sh/). Preconditions: + - `kubectl` with cluster admin access - `helm 3` 1. Add the Helm repo: - ```bash - helm repo add mondoo https://mondoohq.github.io/mondoo-operator - helm repo update - ``` + + ```bash + helm repo add mondoo https://mondoohq.github.io/mondoo-operator + helm repo update + ``` 2. Deploy the operator using Helm: - ```bash - helm install mondoo-operator mondoo/mondoo-operator --namespace mondoo-operator --create-namespace - ``` + ```bash + helm install mondoo-operator mondoo/mondoo-operator --namespace mondoo-operator --create-namespace + ``` ### Installing with Operator Lifecycle Manager (OLM) + Follow these steps to set up the Mondoo Operator using [Operator Lifecycle Manager (OLM)](https://olm.operatorframework.io/): Preconditions: + - kubectl with cluster admin access - [`operator-lifecycle-manager`](https://olm.operatorframework.io/) installed in the cluster (see the [OLM QuickStart](https://olm.operatorframework.io/docs/getting-started/)) - [`operator-sdk`](https://sdk.operatorframework.io/docs/installation/) installed locally - 1. Verify that operator-lifecycle-manager is up: - ```bash - operator-sdk olm status | echo $? - 0 - INFO[0000] Fetching CRDs for version "v0.20.0" - INFO[0000] Fetching resources for resolved version "v0.20.0" - INFO[0001] Successfully got OLM status for version "v0.20.0" - ``` + + ```bash + operator-sdk olm status | echo $? + 0 + INFO[0000] Fetching CRDs for version "v0.20.0" + INFO[0000] Fetching resources for resolved version "v0.20.0" + INFO[0001] Successfully got OLM status for version "v0.20.0" + ``` 2. Install the Mondoo Operator bundle: - ```bash - kubectl create namespace mondoo-operator - operator-sdk run bundle ghcr.io/mondoohq/mondoo-operator-bundle:latest --namespace=mondoo-operator - ``` + + ```bash + kubectl create namespace mondoo-operator + operator-sdk run bundle ghcr.io/mondoohq/mondoo-operator-bundle:latest --namespace=mondoo-operator + ``` 3. Verify that the operator is properly installed: - ```bash - kubectl get csv -n operators - ``` + ```bash + kubectl get csv -n operators + ``` ## Configuring the Mondoo Secret + Follow these steps to configure the Mondoo Secret: + 1. Create a new Mondoo service account to report assessments to [Mondoo Platform](https://mondoo.com/docs/platform/service_accounts). 2. Store the service account json into a local file `creds.json`. The `creds.json` file should look like this: - ```json - { - "mrn": "//agents.api.mondoo.app/spaces//serviceaccounts/", - "space_mrn": "//captain.api.mondoo.app/spaces/", - "private_key": "-----BEGIN PRIVATE KEY-----\n....\n-----END PRIVATE KEY-----\n", - "certificate": "-----BEGIN CERTIFICATE-----\n....\n-----END CERTIFICATE-----\n", - "api_endpoint": "https://api.mondoo.com" - } - ``` + + ```json + { + "mrn": "//agents.api.mondoo.app/spaces//serviceaccounts/", + "space_mrn": "//captain.api.mondoo.app/spaces/", + "private_key": "-----BEGIN PRIVATE KEY-----\n....\n-----END PRIVATE KEY-----\n", + "certificate": "-----BEGIN CERTIFICATE-----\n....\n-----END CERTIFICATE-----\n", + "api_endpoint": "https://api.mondoo.com" + } + ``` 3. Store the service account as a Secret in the Mondoo namespace: - ```bash - kubectl create secret generic mondoo-client --namespace mondoo-operator --from-file=config=creds.json - ``` + ```bash + kubectl create secret generic mondoo-client --namespace mondoo-operator --from-file=config=creds.json + ``` ## Creating a MondooAuditConfig + Once the Secret is configured, configure the operator to define the scan targets: 1. Create `mondoo-config.yaml`: - ```yaml - apiVersion: k8s.mondoo.com/v1alpha2 - kind: MondooAuditConfig - metadata: - name: mondoo-client - namespace: mondoo-operator - spec: - mondooCredsSecretRef: - name: mondoo-client - kubernetesResources: - enable: true - nodes: - enable: true - ``` + + ```yaml + apiVersion: k8s.mondoo.com/v1alpha2 + kind: MondooAuditConfig + metadata: + name: mondoo-client + namespace: mondoo-operator + spec: + mondooCredsSecretRef: + name: mondoo-client + kubernetesResources: + enable: true + nodes: + enable: true + ``` 2. Apply the configuration: - ```bash - kubectl apply -f mondoo-config.yaml - ``` + ```bash + kubectl apply -f mondoo-config.yaml + ``` ### Filter Kubernetes objects based on namespace To exclude specific namespaces add this to your `MondooAuditConfig`: + ``` ... spec: @@ -126,6 +141,7 @@ spec: ``` When you only want to scan specific namespaces: + ``` ... spec: @@ -139,9 +155,11 @@ spec: ``` ## Deploying the admission controller -Kubernetes webhooks require TLS certs to establish the trust between the certificate authority listed in `ValidatingWebhookConfiguration.Webhooks[].ClientConfig.CABundle` and the TLS certificates presented when connecting to the HTTPS endpoint specified in the webhook. + +Kubernetes webhooks require TLS certs to establish the trust between the certificate authority listed in `ValidatingWebhookConfiguration.Webhooks[].ClientConfig.CABundle` and the TLS certificates presented when connecting to the HTTPS endpoint specified in the webhook. You can choose one of three approaches: + - Install and use cert-manager to automate the creation and update of the TLS certs - Use the OpenShift certificate creation/rotation features - Create (and rotate) your own TLS certificates manually @@ -149,11 +167,13 @@ You can choose one of three approaches: A working setup shows the webhook Pod processing the created/modified Pods. Display the logs in one window: + ```bash kubectl logs -f deployment/mondoo-client-webhook-manager -n mondoo-operator ``` And delete a Pod in another window (which will cause a new one to be created): + ```bash kubectl delete pod -n mondoo-operator --selector app.kubernetes.io/name=mondoo-operator ``` @@ -161,6 +181,7 @@ kubectl delete pod -n mondoo-operator --selector app.kubernetes.io/name=mondoo-o ### Scanned workload types Currently, the admission controller can scan these workload types: + - Pods - Deployments - DaemonSets @@ -178,6 +199,7 @@ For more information on how you can configure this, have a look at [this tutoria You can run the admission controller in two modes: permissive and enforcing. You configure the mode via the `MondooAuditConfig`: + ```yaml apiVersion: k8s.mondoo.com/v1alpha2 kind: MondooAuditConfig @@ -191,6 +213,7 @@ You configure the mode via the `MondooAuditConfig`: scanner: replicas: 1 ``` + When admission is enabled, the default mode is `permissive` with one replica. In permissive mode, the webhook checks objects like Deployments or Pods against policies and reports problems to the Mondoo Backend. Mondoo shows the results in the CI/CD view. @@ -200,6 +223,7 @@ The webhook then will deny objects not passing the policy. The details are reported to the Mondoo Backend. With kubectl, this looks similar to this example: + ```bash $ kubectl apply -f ubuntu-privileged.yaml Error from server (FAILED MONDOO SCAN): error when creating "ubuntu-privileged.yaml": admission webhook "policy.k8s.mondoo.com" denied the request: FAILED MONDOO SCAN @@ -209,123 +233,131 @@ Error from server (FAILED MONDOO SCAN): error when creating "ubuntu-privileged.y > > Increase replicas for webhook **and** scanner to at least two. - The Deployments are configured with [`PodAntiAffinity`](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity). This, with a replica count of two, helps to prevent outages because of single Pod or Node failures. Please increase the replicas count according to your needs. - ### Deploying the admission controller using cert-manager -[cert-manager](https://cert-manager.io/) is the easiest way to bootstrap the admission controller TLS certificate: + +[cert-manager](https://cert-manager.io/) is the easiest way to bootstrap the admission controller TLS certificate: 1. Install cert-manger on the cluster if it isn't already installed. ([See instructions](https://cert-manager.io/docs/installation/).) 2. Update MondooAuditConfig so that the webhook section requests TLS certificates from cert-manager: - ```yaml - apiVersion: k8s.mondoo.com/v1alpha2 - kind: MondooAuditConfig - metadata: - name: mondoo-client - namespace: mondoo-operator - spec: - mondooCredsSecretRef: - name: mondoo-client - kubernetesResources: - enable: true - nodes: - enable: true - admission: - enable: true - certificateProvisioning: - mode: cert-manager - ``` + ```yaml + apiVersion: k8s.mondoo.com/v1alpha2 + kind: MondooAuditConfig + metadata: + name: mondoo-client + namespace: mondoo-operator + spec: + mondooCredsSecretRef: + name: mondoo-client + kubernetesResources: + enable: true + nodes: + enable: true + admission: + enable: true + certificateProvisioning: + mode: cert-manager + ``` -The admission controller `Deployment` should start. The`ValidatingWebhookConfiguration` should be annotated to insert the certificate authority data. cert-manager creates a Secret named `webhook-serving-cert` that contains the TLS certificates. +The admission controller `Deployment` should start. The`ValidatingWebhookConfiguration` should be annotated to insert the certificate authority data. cert-manager creates a Secret named `webhook-serving-cert` that contains the TLS certificates. ### Manually creating TLS certificates using OpenSSL + You can manually create the TLS certificate required for the admission controller. These steps show one method: 1. Create a key for your certificate authority: + ```bash openssl genrsa -out ca.key 2048 ``` 2. Generate a certificate for your certificate authority: + ```bash openssl req -x509 -new -nodes -key ca.key -subj "/CN=Webhook Issuer" -days 10000 -out ca.crt ``` 3. Generate a key for the webhook server Pod: + ```bash openssl genrsa -out server.key 2048 ``` 4. Create a config file named `csr.conf` to specify the options for the webhook server certificate. Substitute NAMESPACE with the namespace where the MondooAuditConfig resource was created (ie. mondoo-operator): - ``` - [ req ] - default_bits = 2048 - prompt = no - default_md = sha256 - req_extensions = req_ext - distinguished_name = dn - [ dn ] - C = US - ST = NY - L = NYC - O = My Company - OU = My Organization - CN = Webhook Server - [ req_ext ] - subjectAltName = @alt_names - [ alt_names ] - DNS.1 = mondoo-operator-webhook-service.NAMESPACE.svc - DNS.2 = mondoo-operator-webhook-service.NAMESPACE.svc.cluster.local - [ v3_ext ] - authorityKeyIdentifier=keyid,issuer:always - basicConstraints=CA:FALSE - keyUsage=keyEncipherment,dataEncipherment - extendedKeyUsage=serverAuth,clientAuth - subjectAltName=@alt_names - ``` + + ``` + [ req ] + default_bits = 2048 + prompt = no + default_md = sha256 + req_extensions = req_ext + distinguished_name = dn + [ dn ] + C = US + ST = NY + L = NYC + O = My Company + OU = My Organization + CN = Webhook Server + [ req_ext ] + subjectAltName = @alt_names + [ alt_names ] + DNS.1 = mondoo-operator-webhook-service.NAMESPACE.svc + DNS.2 = mondoo-operator-webhook-service.NAMESPACE.svc.cluster.local + [ v3_ext ] + authorityKeyIdentifier=keyid,issuer:always + basicConstraints=CA:FALSE + keyUsage=keyEncipherment,dataEncipherment + extendedKeyUsage=serverAuth,clientAuth + subjectAltName=@alt_names + ``` 5. Generate the certificate signing request: + ```bash openssl req -new -key server.key -out server.csr -config csr.conf ``` 6. Generate the certificate for the webhook server: + ```bash openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 10000 -extensions v3_ext -extfile csr.conf ``` 7. Create the Secret holding the TLS certificate data: + ```bash kubectl create secret tls -n mondoo-operator webhook-server-cert --cert ./server.crt --key ./server.key ``` 8. Add the certificate authority as base64 encoded CA data (`base64 ./ca.crt`) to the ValidatingWebhookConfiguration under the `webhooks[].clientConfig.caBundle` field: - ```bash - kubectl edit validatingwebhookconfiguration mondoo-operator-mondoo-webhook - ``` + ```bash + kubectl edit validatingwebhookconfiguration mondoo-operator-mondoo-webhook + ``` The end result should resemble this: + ```yaml - apiVersion: admissionregistration.k8s.io/v1 - kind: ValidatingWebhookConfiguration - metadata: - annotations: - mondoo.com/tls-mode: manual - name: mondoo-operator-mondoo-webhook - webhooks: - - admissionReviewVersions: - - v1 - clientConfig: - caBundle: BASE64-ENCODED-DATA-HERE - service: - name: mondoo-operator-webhook-service - namespace: mondoo-operator - path: /validate-k8s-mondoo-com-core - port: 443 +apiVersion: admissionregistration.k8s.io/v1 +kind: ValidatingWebhookConfiguration +metadata: + annotations: + mondoo.com/tls-mode: manual + name: mondoo-operator-mondoo-webhook +webhooks: + - admissionReviewVersions: + - v1 + clientConfig: + caBundle: BASE64-ENCODED-DATA-HERE + service: + name: mondoo-operator-webhook-service + namespace: mondoo-operator + path: /validate-k8s-mondoo-com-core + port: 443 ``` ### Firewall rules for the webhook @@ -341,6 +373,7 @@ Please create a secret with the name `mondoo-private-registries-secrets` within The Mondoo operator will only read a secret with this exact name so that we can limit required RBAC permissions. Please ensure the secret contains access data to all the registries you want to scan. Now add the name to your `MondooAuditConfig` so that the operator knows you want to scan private images. + ```yaml apiVersion: k8s.mondoo.com/v1alpha2 kind: MondooAuditConfig @@ -371,10 +404,13 @@ To do so, follow these steps: 1. Create an additional [Space in Mondoo](https://mondoo.com/docs/platform/spaces/) 2. Create a [Mondoo Service Account](https://mondoo.com/docs/platform/service_accounts/) for this space 3. Create the new namespace in Kubernetes: + ``` kubectl create namespace 2nd-namespace ``` + 4. Create a Kubernetes Service Account in this namespace: + ```yaml apiVersion: v1 kind: ServiceAccount @@ -382,7 +418,9 @@ metadata: name: mondoo-operator-k8s-resources-scanning namespace: 2nd-namespace ``` + 5. Bind this Service Account to a Cluster Role which was created during the installation of the operator: + ```yaml apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding @@ -393,37 +431,45 @@ roleRef: kind: ClusterRole name: mondoo-operator-k8s-resources-scanning subjects: -- kind: ServiceAccount - name: mondoo-operator-k8s-resources-scanning - namespace: 2nd-namespace + - kind: ServiceAccount + name: mondoo-operator-k8s-resources-scanning + namespace: 2nd-namespace ``` + 6. Add the Mondoo Service Account as a secret to the namespace as described [here](https://github.com/mondoohq/mondoo-operator/blob/main/docs/user-manual.md#configuring-the-mondoo-secret) 7. Create a `MondooAuditConfig` in `2nd-namespace` as described [here](https://github.com/mondoohq/mondoo-operator/blob/main/docs/user-manual.md#creating-a-mondooauditconfig) 8. (Optional) In case you want to separate which Kubernetes namespaces show up in which Mondoo Space, you can add [filtering](https://github.com/mondoohq/mondoo-operator/blob/main/docs/user-manual.md#filter-kubernetes-objects-based-on-namespace). After some seconds, you should see that the operator picked up the new `MondooAuditConfig` and starts creating objects. - ## Uninstalling the Mondoo operator + Before uninstalling the Mondoo operator, be sure to delete all `MondooAuditConfig` and `MondooOperatorConfig` objects. You can find any in your cluster by running: + ```bash kubectl get mondooauditconfigs.k8s.mondoo.com,mondoooperatorconfigs.k8s.mondoo.com -A ``` ### Uninstalling the operator with kubectl -Run: + +Run: + ```bash kubectl delete -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml ``` ### Uninstalling the operator with Helm -Run: + +Run: + ```bash helm uninstall mondoo-operator ``` ### Uninstalling the operator with Operator Lifecycle Manager (OLM) -Run: + +Run: + ```bash operator-sdk olm uninstall mondoo-operator ``` @@ -431,6 +477,7 @@ operator-sdk olm uninstall mondoo-operator ## FAQ ### I do not see the service running, only the operator. What should I do? + 1. Check that the CRD is properly registered with the operator: ```bash @@ -448,30 +495,37 @@ mondoo-client 2m44s ``` ### How do I edit an existing operator configuration? + Run: + ```bash kubectl edit mondooauditconfigs -n mondoo-operator ``` ### How do I run asset garbage collection manually? -The operator will trigger garbage collection after each successful cluster scan. This will delete all old assets that were created + +The operator will trigger garbage collection after each successful cluster scan. This will delete all old assets that were created by the operator but are no longer present in the cluster. It is possible to trigger garbage collection manually. To do this it is required to have the Mondoo Client API running locally: + ```bash mondoo serve --api --token abcdefgh ``` Retrieve the cluster UID: + ```bash -kubectl get ns kube-system -o yaml | grep uid +kubectl get ns kube-system -o yaml | grep uid ``` Then you can trigger asset garbage collection for workloads by the following command: + ```bash mondoo-operator garbage-collect --filter-managed-by mondoo-operator-<> --filter-older-than 2h --filter-platform-runtime k8s-cluster --scan-api-url http://127.0.0.1:8989 --token abcdefgh ``` For container images use: + ```bash mondoo-operator garbage-collect --filter-managed-by mondoo-operator-<> --filter-older-than 48h --filter-platform-runtime docker-registry --scan-api-url http://127.0.0.1:8989 --token abcdefgh ``` @@ -479,6 +533,7 @@ mondoo-operator garbage-collect --filter-managed-by mondoo-operator-<-node--hash`. For example, if your `MondooAuditConfig` is called `mondoo-client` and you have a node called `node01`, you should be able to find a pod `mondoo-client-node-node01-`. +Look for pods in the form of `-node--hash`. For example, if your `MondooAuditConfig` is called `mondoo-client` and you have a node called `node01`, you should be able to find a pod `mondoo-client-node-node01-`. -If the pod is crashing and restarting, it's most probably running out of memory and terminating. You can verify that by looking into the pod's status: +If the pod is crashing and restarting, it's most probably running out of memory and terminating. You can verify that by looking into the pod's status: ```bash kubectl get pods -n mondoo-operator mondoo-client-node-node01- -o yaml ``` If you need to increase the resource limits for node scanning, change your `MondooAuditConfig`: + ```bash kubectl edit -n mondoo-operator mondooauditconfig mondoo-client ``` Search for the `nodes:` section and specify the new limits there. It should look like this: + ```yaml spec: nodes: @@ -529,95 +588,116 @@ spec: ``` ### How can I trigger a new scan? + The operator runs a full cluster scan and node scans hourly. If you need to manually trigger those scans there are two options: Option A: Create a job from the existing cron job - 1. Locate the cron job you want to trigger: - ```bash - kubectl get cronjobs -n mondoo-operator - ``` - 2. Create a new job from the existing cron job. To trigger a new cluster scan, the command is: - ```bash - kubectl create job -n mondoo-operator my-job --from=cronjob/mondoo-client-k8s-scan - ``` - 3. A job called `my-job` starts the scan immediately. + +1. Locate the cron job you want to trigger: + +```bash +kubectl get cronjobs -n mondoo-operator +``` + +2. Create a new job from the existing cron job. To trigger a new cluster scan, the command is: + +```bash +kubectl create job -n mondoo-operator my-job --from=cronjob/mondoo-client-k8s-scan +``` + +3. A job called `my-job` starts the scan immediately. Option B: Turn scanning off and then on again - 1. Edit the `MondooAuditConfig`: - ```bash - kubectl edit -n mondoo-operator mondooauditconfig mondoo-client - ``` - 2. Disable scanning by changing `enable: true` to `enable: false`: - ```yaml - spec: - kubernetesResources: - enable: false - nodes: - enable: false - ``` - 3. Make sure the scan cron jobs are deleted before proceeding: - ```bash - kubectl get cronjobs -n mondoo-operator - ``` - 4. Edit the `MondooAuditConfig` again and re-enable scanning: - ```yaml - spec: - kubernetesResources: - enable: true - nodes: - enable: true - ``` - 5. The scan cron jobs will be re-created and their initial run will occur within the next minute. + +1. Edit the `MondooAuditConfig`: + +```bash +kubectl edit -n mondoo-operator mondooauditconfig mondoo-client +``` + +2. Disable scanning by changing `enable: true` to `enable: false`: + +```yaml +spec: + kubernetesResources: + enable: false + nodes: + enable: false +``` + +3. Make sure the scan cron jobs are deleted before proceeding: + +```bash +kubectl get cronjobs -n mondoo-operator +``` + +4. Edit the `MondooAuditConfig` again and re-enable scanning: + +```yaml +spec: + kubernetesResources: + enable: true + nodes: + enable: true +``` + +5. The scan cron jobs will be re-created and their initial run will occur within the next minute. ### I had a `MondooAuditConfig` in my cluster with version `v1alpha1` and now I can no longer access it. What should I do? + Mondoo recently upgraded our CRDs version to `v1alpha2`. You need to manually migrate to the new version. You can list the CRDs with the old version by running: + ```bash kubectl get mondooauditconfigs.v1alpha1.k8s.mondoo.com -A ``` -Manually edit each of the CRDs in the list to map it to the new version. +Manually edit each of the CRDs in the list to map it to the new version. -Note: This is not possible immediately after performing the operator upgrade. +Note: This is not possible immediately after performing the operator upgrade. 1. Back up your old `MondooAuditConfig`: - ```bash - kubectl get mondooauditconfigs.v1alpha1.k8s.mondoo.com mondoo-client -n mondoo-operator -o yaml > audit-config.yaml - ``` + + ```bash + kubectl get mondooauditconfigs.v1alpha1.k8s.mondoo.com mondoo-client -n mondoo-operator -o yaml > audit-config.yaml + ``` 2. Map the old `v1alpha1` config to the new `v1alpha2` and save the new `MondooAuditConfig`. Find the mapping from `v1alpha1` to `v1alpha2` [here](../api/v1alpha1/mondooauditconfig_types.go#L155-L199). 3. Disable the `webhook` conversion for the `MondooAuditConfig` CRD: - ```bash - kubectl edit crd mondooauditconfigs.k8s.mondoo.com - ``` - Delete or comment out this section: - ```yaml - spec: - # conversion: - # strategy: Webhook - # webhook: - # clientConfig: - # service: - # name: webhook-service - # namespace: mondoo-operator - # path: /convert - # conversionReviewVersions: - # - v1 - group: k8s.mondoo.com - names: - kind: MondooAuditConfig - listKind: MondooAuditConfigList - plural: mondooauditconfigs - singular: mondooauditconfig - ``` + ```bash + kubectl edit crd mondooauditconfigs.k8s.mondoo.com + ``` + + Delete or comment out this section: + + ```yaml + spec: + # conversion: + # strategy: Webhook + # webhook: + # clientConfig: + # service: + # name: webhook-service + # namespace: mondoo-operator + # path: /convert + # conversionReviewVersions: + # - v1 + group: k8s.mondoo.com + names: + kind: MondooAuditConfig + listKind: MondooAuditConfigList + plural: mondooauditconfigs + singular: mondooauditconfig + ``` 4. Apply the updated `MondooAuditConfig`: - ```bash - kubectl apply -f audit-config.yaml - ``` + + ```bash + kubectl apply -f audit-config.yaml + ``` 5. Restore the original CRD definition. The easiest way to do that is to apply the manifests from our latest release: - ```bash - kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml - ``` + ```bash + kubectl apply -f https://github.com/mondoohq/mondoo-operator/releases/latest/download/mondoo-operator-manifests.yaml + ```