Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updates K8s node scanning documentation #500

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

scottford-io
Copy link
Contributor

Description

image

From @imilchev :

the mondoo-operator uses filesystem scanning to scan the k8s nodes. That cannot check all the stuff we normally check. On the container-optimized operating systems we may be locked out of having access to /proc or to the root filesystem. This means some checks may results in ERROR results. This is what is happening in lunalectric. The operator gets ERROR results for some checks, which pass when scanning the same node from AWS Serverless integration. You get flopping check results.
There is also a bigger problem than just this. Mondoo is no configured to scan the same asset from 2 different integrations. We never supported that and we still don’t. If the 2 integrations happen to scan the asset at the same time in parallel, you are definitely getting weird results in the UI. On top of that, if you open an asset, you can see which integration “owns” the asset. If you scan the same asset from multiple integrations, that information will be inaccurate. One time it may say the asset is owned by integration A and the next time it may be integration B. Asset overview table also gets different info based on what integration scanned the asset. In your case the asset name also changes because the mondoo-operator uses the node name in the cluster as asset name. However, AWS Serverless is using the EC2 name as asset name.
Bottom line is, you should NOT scan the same thing with different integrations. That is only opening the door for all kinds of weird results. On top of that, at the moment, we don’t offer any additional value if you try to do this. You are just performing unnecessary work (because you are scanning something you already scanned again). If you have AWS serverless integration running, which scans EC2 instances, there is no benefit and no point at all to use node scanning with the mondoo-operator
All clouds. It’s the same concept. It’s similar to scanning a linux server with a cnspec service but also running an ssh scan from somewhere else. It really gives you no extra info, you are just scanning the same thing twice

Related issue

Types of changes

  • Functional documentation bug fix (i.e., broken link or some other busted behavior)
  • New functional doc capabilities (i.e., filter search results)
  • New content
  • Revision to existing content
  • Chore (non-breaking change that does not add functionality or fix an issue)

Checklist

  • I have read the README document about contributing to this repo.
  • I have tested my changes locally and there are no issues.
  • All commits are signed.

Copy link
Contributor

Starting creation of the PREview environment...

Copy link
Contributor

PREview environment has been created at https://mondoo-pre-docs-2010572077.storage.googleapis.com/docs/index.html
Please allow a few minutes for the environment to be fully deployed.

Copy link
Contributor

@misterpantz misterpantz marked this pull request as draft December 18, 2024 00:31
@misterpantz
Copy link
Contributor

Converting to draft until @imilchev reviews and I make a change for clarity.

Updating wording per Ivan

Signed-off-by: scottford-io <[email protected]>
@scottford-io scottford-io force-pushed the scottford/k8s-node-scanning branch from 36e6080 to 505811a Compare December 18, 2024 15:36
Copy link
Contributor

@scottford-io
Copy link
Contributor Author

@misterpantz I think this one is GTG.


- We strongly recommend that you leave **CronJob-based** selected. It's ideal for most infrastructures. A CronJob executes regularly to run the scans without permanently allocating any resources for Mondoo on cluster nodes.
Mondoo can scan both a Kubernetes (K8s) cluster using the Mondoo K8s Operator as well as the account (AWS account, GCP project, or Azure subscription) where the cluster is deployed. To avoid duplication of assets, if the account is integrated with VM scanning enabled, or if you plan to enable it, ensure that Node Scanning is disabled for the Kubernetes cluster.
Copy link
Contributor

@misterpantz misterpantz Dec 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Mondoo can scan both a Kubernetes (K8s) cluster using the Mondoo K8s Operator as well as the account (AWS account, GCP project, or Azure subscription) where the cluster is deployed. To avoid duplication of assets, if the account is integrated with VM scanning enabled, or if you plan to enable it, ensure that Node Scanning is disabled for the Kubernetes cluster.
Mondoo can scan Kubernetes clusters in two ways:
- Directly, using the Mondoo Kubernetes Operator
- As part of the cloud account (AWS account, GCP project, or Azure subscription) where the cluster is deployed
If you enable (or plan to enable) VM scanning when you set up your AWS, GCP, or Azure integration AND you enable **Scan nodes** in your Kubernetes integration, you create duplicate assets (two of the same cluster). To prevent this, be sure to disable **Scan nodes** in the Kubernetes integration.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not that easy to understand, I'm afraid. Partly just because it's a hard thing to describe but also because (1) the setting name is different than in the UI and (2) the structure.
Does my suggestion help without mucking up the meaning?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@misterpantz I think your change is drastically better! 🙌 🥳

I have slight tweaks that you can take or leave:

Suggested change
Mondoo can scan both a Kubernetes (K8s) cluster using the Mondoo K8s Operator as well as the account (AWS account, GCP project, or Azure subscription) where the cluster is deployed. To avoid duplication of assets, if the account is integrated with VM scanning enabled, or if you plan to enable it, ensure that Node Scanning is disabled for the Kubernetes cluster.
Mondoo can scan Kubernetes clusters in two ways:
- **Directly**, using the Mondoo Kubernetes Operator.
- **Indirectly**, as part of the cloud account (AWS account, GCP project, or Azure subscription) where the cluster is deployed.
If both VM scanning is enabled for your cloud account integration and Scan nodes is enabled for your Kubernetes integration, this will create duplicate assets (the same cluster appearing twice). To avoid duplication, disable Scan nodes in your Kubernetes integration.

The only other thought I had would be to add something explaining Scan nodes:

What is “Scan nodes”?

This setting allows Mondoo to scan Kubernetes cluster nodes directly. If you’re scanning the cloud account, the nodes will already be scanned as part of VM scanning.

For steps to disable Scan nodes, see [link to instructions].

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because you're adding this note WITHIN the instructions, right below the step that tells you to enable Scan nodes if you want to assess the security posture of nodes in your Kubernetes cluster, there's no need to link to it. :)

And I don't want to reintroduce the passive voice. Active voice makes it clear that you/your team is doing the enabling.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has some of your suggestions and some of mine:

Suggested change
Mondoo can scan both a Kubernetes (K8s) cluster using the Mondoo K8s Operator as well as the account (AWS account, GCP project, or Azure subscription) where the cluster is deployed. To avoid duplication of assets, if the account is integrated with VM scanning enabled, or if you plan to enable it, ensure that Node Scanning is disabled for the Kubernetes cluster.
Mondoo can scan Kubernetes clusters in two ways:
- Directly, using the Mondoo Kubernetes Operator
- Indirectly, as part of the cloud account (AWS account, GCP project, or Azure subscription) where the cluster is deployed
If you enable **Scan virtual machines** in your cloud account integration AND your enable **Scan nodes** in your Kubernetes integration, Mondoo creates duplicate assets (the same cluster represented twice). To prevent duplication, disable **Scan nodes** in your Kubernetes integration.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh crap... I just checked and, of the cloud integrations you listed, the only one that has a Scan virtual machines option is Azure!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants