Need help understanding WARN state and scoring mechanism

[Docteur-RS]

Hey !

I'm a bit confused about the [WARN] state...

Can someone explain the difference between "scored" and "not scored" ?

I also don't get how it influences the state ? Scored + unable to test creates what in the end ??
And the other way around is not clear either. The worst case that blows each theories that I've got so far is the warn + scrored. This makes no sense what so ever ! :-p

Thanks in advanced

[lizrice]

"Scored" or "Not scored" comes from the CIS benchmark itself (so if you find any cases where our test files don't match the CIS benchmark, please raise an issue).

"Warn" means that the test needs manual attention: see https://github.com/aquasecurity/kube-bench#output

kube-bench might be unable to run a test because, for example, an executable that it needs to run is not available in the path.

Does that help? Open to ideas on how to clarify the README

[Docteur-RS]

Thanks for your answer.

I got the following From the CIS documentation :

Scored

• Failure to comply with "Scored" recommendations will decrease the final benchmark score.
  Compliance with "Scored" recommendations will increase the final benchmark score.

Not Scored

• Failure to comply with "Not Scored" recommendations will not decrease the final
  benchmark score. Compliance with "Not Scored" recommendations will not increase the
  final benchmark score.

This could be part of the readme because

• The "scored" and "not scored" status appears everywhere in the output of kube-bench
• There is no explanation of the terms "scored" and "not scored" in the Readme
• The CIS definition of the status is quite hard to find on the internet.

---

There are three output states:

[PASS] and [FAIL] indicate that a test was run successfully, and it either passed or failed.
[WARN] means this test needs further attention, for example it is a test that needs to be run manually.
[INFO] is informational output that needs no further action.
Note:

If the test is Manual, this always generates WARN (because the user has to run it manually)
If the test is Scored, and kube-bench was unable to run the test, this generates FAIL (because the test has not been passed, and as a Scored test, if it doesn't pass then it must be considered a failure).
If the test is Not Scored, and kube-bench was unable to run the test, this generates WARN.
If the test is Scored, type is empty, and there are no test_items present, it generates a WARN.

Let's decompose the above paragraph taken from the readme:

PASS

• Test was able to run + Test has PASSED => What about scored and not scored in this state ?

WARN

• (A) May have to be run manually => What about scored and not scored ?
• (B) Runned automatically + "not scored" + unable to run the test => Lacks information in readme about why a test could not be run

So I guess that all WARN + "scored" are type (A) because (B) is "not scored" only. This should also be in the Readme.

FAIL

• Test was able to run + Test has FAILED
• Runned automatically + "scored" + unable to run the test

The readme could use a board to sum up all the different cases.

---

Lets take an example:
Test:
1.1.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)

Remediation:
1.1.9 Run the below command (based on the file location on your system) on the master node. For example, chmod 644 <path/to/cni/files>

So, WARN + "not scored" is type (B). Then it should mean that the test was not able to run.
Why is a simple chmod check was unable to run ?

Second example:
Test:
[WARN] 1.1.19 Ensure that the Kubernetes PKI directory and file ownership is set to root:root (Scored)

Remediation:
1.1.19 Run the below command (based on the file location on your system) on the master node. For example, chown -R root:root /etc/kubernetes/pki/
So, WARN + "scored" means type (A). Then this should be a manual test... Why isn't this automatic ? And if it must be run manualy, what should I do ? Should I make my own test based on the description of the "test" section ?

---

A lot of questions here ! I hope that you can lift a few of my interogation ! 😄

[lizrice]

Just to make things more complicated, in v1.6.0 of the CIS benchmark, Scored / Not Scored is replaced by Automated / Manual. This largely makes sense because mostly the old Scored tests were ones that could be automated, because it was felt that it's difficult to require a Score on a test that has to be run manually.

For 1.1.9 the problem is knowing what the file location is for the CNI file. Once you know where the file it, it is easy to automate checking the permissions. In the new version of the spec this is marked as a Manual test.

It is totally fine to modify your own version of the test to automate it.

[lizrice]

In #691 I'm adding some clarification - the main point being that if you see FAIL or WARN, the remediation output for that test should describe what you need to do.

[lizrice]

I think this was resolved by #691 so I'm going to mark as answered

[Josue198s]

What about [INFO] what doees it means?

[chen-keinan]

when the test type is marked for skip then it state becomes INFO , meaning if you wish to skip tests

[etcshad0vv]

Hello,

Based on https://www.cisecurity.org/insights/blog/changes-to-cis-benchmark-assessment-recommendation-scoring the changes when it comes to published documents, Automated=Scored, Manual=Not Scored, i noticed that some of the CIS profiles are not really reflecting this properly,

e.g.

• 4.2.11 > https://github.com/aquasecurity/kube-bench/blob/3f85968c3c65e29fd3d534c6de52d5554449724b/cfg/cis-1.20/node.yaml#L415

i found few more on section 5 about Policies, where some of the controls are supposed to be Automated/Scored.

Is my understanding correct here or kube-bench has another explanation of what is considered scored or not?

[mozillazg]

@etcshad0vv

Is my understanding correct here or kube-bench has another explanation of what is considered scored or not?

We will follow the "Automated=scored" rule when we can audit the tests via a shell script,
we always follow the "Manual=Not Scored" rule.

More context about "Automated" and "Manual":

#906 (comment)
#772 (comment)

[etcshad0vv]

That i get, but not sure if for instance in benchmark cis-1.20 the control 4.2.11 was intentionally left as not-scored although in the CIS published document is Automated and in the kube-bench profile, it has some audit test. That is what confused me.
I do understand that some of the controls on section 5 related to Policies are type: manual which are not scored in case of kube-bench, but they are in case of upstream CIS published document.

So in first case, perhaps it was just missed, and in second case it was intentionally left as not-scored because all the controls there are type: manual. Can you confirm?

[mozillazg]

So in first case, perhaps it was just missed

The control 4.2.11 in benchmark cis-1.20 should be Automated. Look's like it was copied from cis-1.6 and that is a bug. Thanks for your reporting! I created an issue for this bug: #1203

In second case it was intentionally left as not-scored because all the controls there are type: manual

Yes, you are right.