[BUG] subject_root variable missing

[sluetze]

./scs-compliance-check.py -v -a kubeconfig=path/to/kubeconfig.yaml -s SUBJECT scs-compatible-kaas.yaml

does not work

./scs-compliance-check.py -v -a kubeconfig=${HOME}/.kube/config -s OCP4.16 scs-compatible-kaas.yaml
CRITICAL: Missing variable assignments (via -a) for: subject_root
Traceback (most recent call last):
  File "/home/sluetzen/scs/standards/Tests/./scs-compliance-check.py", line 382, in <module>
    sys.exit(main(sys.argv[1:]))
  File "/home/sluetzen/scs/standards/Tests/./scs-compliance-check.py", line 339, in main
    raise RuntimeError(f"Missing variable assignments (via -a) for: {', '.join(missing_vars)}")
RuntimeError: Missing variable assignments (via -a) for: subject_root

The Readme says to use it this way

https://github.com/SovereignCloudStack/standards/blob/main/Tests/README.md?plain=1#L34-L44

Running ./scs-compliance-check.py -v -a subject_root=./kubeconfig -s OCP4.16 scs-compatible-kaas.yaml
where ./kubeconfig is a folder having kubeconfig.yaml works

[garloff]

@mbuechse : Doc bug or code bug ?

[mbuechse]

@garloff doc bug

[mbuechse]

@garloff On second thought... Technically, it's a doc bug, but it's also just a questionable UX. I think I will try to find a way to make the UX better as well.

[sluetze]

another thing (maybe a new issue?) the cncf compatibility checks are not done by the script for now. I do not know it this is expected (as the result says failed). Also I am missing any docs on how to run the cncf compatibility checks with the scs tooling.

[mbuechse]

@sluetze They are supposed to be run. Sonobuoy has to be in the path. Can you attach some detailed console output?

[sluetze]

Topic better UX: would be great if the tool would just use the KUBECONFIG env or look in default paths

The KUBECONFIG environment variable holds a list of kubeconfig files. For Linux and Mac, the list is colon-delimited. For Windows, the list is semicolon-delimited. The KUBECONFIG environment variable is not required. If the KUBECONFIG environment variable doesn't exist, kubectl uses the default kubeconfig file, $HOME/.kube/config.

just needed to login for an updated kubeconfig, then rename it to kubeconfig.yaml...

relevant line was: INFO: module cncf-k8s-conformance missing checks or test cases

since there have been an update to the yaml file in between I updated my local copy

after git pull the issue changed to junitparser not found
after adding junitparser==3.2.0 from kaas/requirements.txt to test-requirements.txt and pip install -r test-requirements.txt the error shows sonobuoy executable missing.

So disregard that for now. I ll try to get it running with sonobuoy first.

[mbuechse]

Thanks for going through the ordeal. It's still wet behind the ears, and it does have infant diseases.

[sluetze]

no problem. this is opensource on the master branch, no paid product with enterprise SLAs ;-).

appreciate the fast responses. As I am firing against a non-vanilla-kubernetes with a couple of deviations from upstream, I expect to run into challenges.

[garloff]

OK, I'm happy to review suggestions how to deal with this.
I have seen #838 and I do think it is the minimal doc fix required here -- just waiting for this discussion to come to a conclusion before approving ...
If you agree on instead changing the code to improve usability, please create a separate PR that I can look at.
Thanks!