Extend implementation notes

Add and extend some implementation notes derived from the findings of #426.

Signed-off-by: Hannes Baum <[email protected]>

Standards/scs-0100-w1-flavor-naming-implementation-testing.md

@@ -15,21 +15,20 @@ The three major versions of the standard that exist so far are very similar, and
 Therefore, the procedures needed to implement or test them are very similar as well. Yet, this document
 will only cover v3, because v1 and v2 are already obsolete by the time of writing.
 
-## Implementation Notes
+## Implementation notes
 
 Every flavor whose name starts with `SCS-` must conform with the naming scheme laid down in the standard.
 
-### Operational Tooling
+### Operational tooling
 
-#### Syntax Check
+#### Syntax check
 
 The [test suite](https://github.com/SovereignCloudStack/standards/tree/main/Tests/iaas/flavor-naming)
 comes with a handy
 [command-line utility](https://github.com/SovereignCloudStack/standards/tree/main/Tests/iaas/flavor-naming/cli.py)
-that can be used to validate flavor names, to
-interactively construct a flavor name via a questionnaire, and to generate prose descriptions for given
-flavor names. See the
-[README](https://github.com/SovereignCloudStack/standards/tree/main/Tests/iaas/flavor-naming/README.md)
+that can be used to validate flavor names, to interactively construct a flavor name
+via a questionnaire, and to generate prose descriptions for given flavor names.
+See the [README](https://github.com/SovereignCloudStack/standards/tree/main/Tests/iaas/flavor-naming/README.md)
 for more details.
 
 The functionality of this script is also (partially) exposed via the web page
@@ -39,12 +38,19 @@ With the OpenStack tooling (`python3-openstackclient`, `OS_CLOUD`) in place, you
 `cli.py -v parse v3 $(openstack flavor list -f value -c Name)` to get a report
 on the syntax compliance of the flavor names of the cloud environment.
 
-#### Flavor Creation
+#### Flavor creation
 
-The [OpenStack Flavor Manager](https://github.com/osism/openstack-flavor-manager) will create a whole set
-of flavors in one go, given a YAML description of this set.
+The [OpenStack Flavor Manager from osism](https://github.com/osism/openstack-flavor-manager)
+will create a set of flavors.
+The manager provides different options for flavor creation; either the standard mandatory and
+possibly recommended flavors can be created or the user can set a file containing his flavors.
 
-## Automated Tests
+If a CSP wants to provide their own flavors with the `SCS` naming schema, a tool like
+the ["Flavor Site"](https://flavors.scs.community) could be used to reduce time spent on creating
+flavors with the correct naming. These names could then be transferred to the Openstack Flavor Manager
+for easier creation.
+
+## Automated tests
 
 ### Errors
 
@@ -68,6 +74,6 @@ The script [`flavor-names-openstack.py`](https://github.com/SovereignCloudStack/
 talks to the OpenStack API of the cloud specified by the `OS_CLOUD` environment and queries properties and
 checks the names for standards compliance.
 
-## Manual Tests
+## Manual tests
 
 To be determined.

Standards/scs-0101-w1-entropy-implementation-testing.md

@@ -7,18 +7,33 @@ supplements:
   - scs-0101-v1-entropy.md
 ---
 
-## Implementation
+## Introduction
+
+The entropy standard mainly defines recommended flavor attributes to be available and
+also requires a high enough and consistent entropy to be available for all possible flavors.
+This is primarily tested with "FIPS 140-2 failures" as well as the entropy count available
+under `/proc/sys/kernel/random/entropy_avail`.
+
+## Implementation notes
 
 We presume that almost nothing has to be done (or indeed can be done), as
-long as the CPUs and VM images are reasonably recent; only the flavor and
-image attributes have to be set:
+long as the CPUs and VM images are reasonably recent.
+The already mentioned failure testing and entropy count is available
+on modern linux kernels (especially on 5.18 or higher),
+but if older kernels are made available, the CSP needs to provide entropy
+by offering CPU instructions that provide entropy without being filtered by the hypervisor.
+All newer CPUs offer instructions for this kind of use-case (e.g. Intel introduced RDRAND
+in 2014 in their Broadwell architecture) and the expectation is,
+that most CSPs don't use older processors.
+
+Most times, only the flavor and image attributes required by the standard have to be set:
 
 - flavor: `hw_rng:allowed=True` ,
 - image: `hw_rng_model: virtio` .
 
-## Automated Tests
+## Automated tests
 
-### Images Sample
+### Images sample
 
 Some checks need to be performed on a live instance. For these checks, it is
 necessary to choose a sample of VM images to test on.
@@ -59,6 +74,6 @@ as ensured by the image metadata standard.
 The script [`entropy-check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/entropy/entropy-check.py)
 connects to OpenStack and performs the checks described in this section.
 
-## Manual Tests
+## Manual tests
 
 None.

Standards/scs-0102-w1-image-metadata-implementation-testing.md

@@ -0,0 +1,52 @@
+---
+title: "SCS Image Metadata: Implementation and Testing Notes"
+type: Supplement
+track: IaaS
+status: Proposal
+supplements:
+  - scs-0102-v1-image-metadata.md
+---
+
+## Introduction
+
+The standard defines a set of metadata properties for public images, that need to be set in order
+to have an SCS-compliant IaaS setup.
+
+## Implementation notes
+
+The [OpenStack Image Manager from osism](https://github.com/osism/openstack-image-manager)
+will create a set of images from a file provided by the user, which can also set the required properties
+for these images. Additional images with their respective properties besides the images mandatory
+through the ["SCS Standard Images Standard (scs-0104-v1-standard-images)"](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0104-v1-standard-images.md) can also be defined
+this way.
+
+The SCS project provides a [file](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/SCS-Spec.Images.yaml) derived from the [`scs-0104-v1-images.yaml`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/scs-0104-v1-images.yaml) with the necessary
+properties.
+This process also enables the easy adoption of the
+["SCS Standard Images Standard (scs-0104-v1-standard-images)"](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0104-v1-standard-images.md).
+
+## Automated tests
+
+### Images sample
+
+Some checks need to be performed on a live instance. All publicly available images on this instance
+will be checked for either only the mandatory properties or possibly also the recommended ones.
+Additionally, a user can also decide to test their private images, although this isn't a necessity.
+
+### Errors and warnings
+
+The test provides increases its return value for every error found during execution.
+Additionally, logs are provided to output further information:
+
+- Error   for invalid property values or missing properties
+- Warning for missing images or not recommended values
+- Info    for violations of recommendations
+
+### Implementation
+
+The script [`image-md-check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/image-metadata/image-md-check.py)
+connects to OpenStack and performs the checks described in this section.
+
+## Manual tests
+
+None.

Standards/scs-0104-w1-standard-images-implementation-testing.md

@@ -0,0 +1,47 @@
+---
+title: "SCS Standard Images: Implementation and Testing Notes"
+type: Supplement
+track: IaaS
+status: Proposal
+supplements:
+  - scs-0104-v1-standard-images.md
+---
+
+## Introduction
+
+The standard defines a set of images with specified names and properties.
+
+## Implementation notes
+
+The [OpenStack Image Manager from osism](https://github.com/osism/openstack-image-manager)
+will create a set of images from a file provided by the user.
+The SCS project provides such a [file](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/SCS-Spec.Images.yaml) derived from the [`scs-0104-v1-images.yaml`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/scs-0104-v1-images.yaml), which is
+defined by the standard to contain the mandatory and recommended images.
+This also enables the easy adoption of the ["SCS Image Metadata Standard (scs-0102-v1-image-metadata)"](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0102-v1-image-metadata.md).
+
+## Automated tests
+
+### Images sample
+
+Some checks need to be performed on a live instance. For these checks, the [`scs-0104-v1-images.yaml`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/scs-0104-v1-images.yaml)
+file is used as a default to check the images provided on the IaaS instance
+against the expected set of images.
+
+### Errors and warnings
+
+The test provides a return value of up to 127, depending on the number of errors that occurred
+during testing. Additionally, logs are provided to provide further information:
+
+- CRITICAL   for problems preventing the test to complete,
+- ERROR      for violations of requirements,
+- WARNING    for violations of recommendations,
+- DEBUG      for background information and problems that don't hinder the test.
+
+### Implementation
+
+The script [`images-openstack.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/standard-images/images-openstack.py)
+connects to OpenStack and performs the checks described in this section.
+
+## Manual tests
+
+None.

Standards/scs-0210-w1-k8s-version-policy-implementation-testing.md

@@ -0,0 +1,61 @@
+---
+title: "SCS K8S Version Policy: Implementation and Testing Notes"
+type: Supplement
+track: KaaS
+status: Proposal
+supplements:
+  - scs-0210-v1-k8s-new-version-policy.md
+  - scs-0210-v2-k8s-version-policy.md
+---
+
+## Introduction
+
+The standard [SCS K8s version Policy](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0210-v2-k8s-version-policy.md) is in its second iteration and sets the time windows
+for K8s versions to be supported in an SCS context as well as update policies for K8s clusters.
+
+All of this just breaks down to providing new versions in a KaaS offering in a timely fashion
+(depending on versions) and also providing version support for as long as the versions
+are officially supported by Kubernetes.
+
+## Implementation notes
+
+A CSP must make new versions for their KaaS offering available in a timely fashion, so that
+new versions are available in a short window of time.
+Older versions need to be supported until the end of their support window.
+
+Concrete implementation details can't be give here, since not every CSP does provide
+their versions the same way. The best advice to give is to monitor the
+[Kubernetes releases page](https://kubernetes.io/releases/) closely.
+
+## Automated tests
+
+### Notes
+
+The test for the [K8s Version Policy Standard](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0210-v2-k8s-version-policy.md)
+can't be used like most other tests provided in the SCS standards repository.
+It aims on testing a KaaS offering of a CSP with the creation of the most recent
+Kubernetes version provided by the CSP. Since only this part is tested,
+the test doesn't have any validity for a single cluster, since the updating and
+versioning should be decided by the user, which could want to leave a server
+on a version just for stability reasons.
+
+### Errors and warnings
+
+The test will return 0 precisely when it could be verified that the standard is satisfied.
+Otherwise, the return code is the number of errors that occurred (up to 127 due to OS
+restrictions); additionally, the following log messages can occur:
+
+- CRITICAL   for problems preventing the test to complete,
+- ERROR      for violations of requirements,
+- INFO       for violations of recommendations,
+- DEBUG      for background information and problems that don't hinder the test.
+
+### Implementation
+
+The script [`k8s_version_policy.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/kaas/k8s-version-policy/k8s_version_policy.py)
+connects to an existing K8s cluster and checks the version against a list of versions, that
+are calculated to be inside a recency window.
+
+## Manual tests
+
+None.

Standards/scs-0211-w1-kaas-default-storage-class-implementation-testing.md

@@ -0,0 +1,56 @@
+---
+title: "SCS KaaS default storage class: Implementation and Testing Notes"
+type: Supplement
+track: KaaS
+status: Proposal
+supplements:
+  - scs-0211-v1-kaas-default-storage-class.md
+---
+
+## Introduction
+
+The standard [SCS Kaas Default storage class](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0211-v1-kaas-default-storage.md) wants to define the properties of a default `StorageClass` a Kubernetes cluster would rely on,
+if a `PersistentVolumeClaim` doesn't provide a name for one during its creation.
+
+## Implementation notes
+
+A `StorageClass` is made default by using the `storageclass.kubernetes.io/is-default-class`
+annotation; a standardized name is not given. `ReadWriteOnce` must be supported by the volume,
+and it must be protected against data loss due to hardware failures.
+Therefore, volumes must not be bound to the lifecycle of a Kubernetes node and, at best,
+be backed by some kind of redundant storage.
+Guarantees for latency, bandwidth, IOPS and so on are not given.
+
+The cost-intensive part of this standard would be the hardware failure protection by binding
+the `StorageClass` to redundant, non-lifecycle bound storage, since this would mean that
+storage needs to be provided in a higher capacity to achieve the same usable capacity.
+
+## Automated tests
+
+### Notes
+
+The test for the [SCS Kaas Default storage class](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0211-v1-kaas-default-storage.md)
+checks if a default storage class is available and if this storage class can be used
+to create a `PersistentVolume` from a `PersistentVolumeClaim` for a container.
+
+### Errors and warnings
+
+The test will return 0 precisely when it could be verified that the standard is satisfied.
+Otherwise, different return codes are provided depending on the type of error.
+These are as follows (taken from the test script):
+
+- 1    Not able to connect to k8s api
+- 31   Default storage class has no provisioner
+- 32   None or more than one default Storage Class is defined
+- 41   Not able to bind PersistentVolume to PersistentVolumeClaim
+- 42   ReadWriteOnce is not a supported access mode
+
+### Implementation
+
+The script [`k8s-default-storage-class-check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/kaas/k8s-default-storage-class/k8s-default-storage-class-check.py)
+connects to an existing K8s cluster and checks for the availability of a default storage class.
+This can also be done via Sonobuoy.
+
+## Manual tests
+
+None.

Standards/scs-0214-w1-k8s-node-distribution-implementation-testing.md

@@ -0,0 +1,55 @@
+---
+title: "Kubernetes Node Distribution and Availability: Implementation and Testing Notes"
+type: Supplement
+track: KaaS
+status: Proposal
+supplements:
+  - scs-0214-v1-k8s-node-distribution.md
+  - scs-0214-v2-k8s-node-distribution.md
+---
+
+## Introduction
+
+The standard [SCS K8s Node Distribution and Availability](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0214-v2-k8s-node-distribution.md)
+tries to define requirements for the distribution of Kubernetes nodes in order to provide
+a fault-tolerant and (highly) available Kubernetes cluster.
+
+## Implementation notes
+
+A Kubernetes clusters control plane must be distributed over multiple physical machines, as well
+as different "failure zones". How these are defined is at the moment up to the CSP.
+Worker nodes can also be distributed over "failure zones", but this isn't a requirement.
+Distribution must be shown through labelling, so that users can access these information.
+
+Node distribution metadata is provided through the usage of the labels
+`topology.kubernetes.io/region`, `topology.kubernetes.io/zone` and
+`topology.scs.community/host-id` respectively.
+
+At the moment, not all labels are set automatically by most K8s cluster utilities, which incurs
+additional setup and maintenance costs.
+
+## Automated tests
+
+### Notes
+
+The test for the [SCS K8s Node Distribution and Availability](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0214-v2-k8s-node-distribution.md)
+checks if control-plane nodes are distributed over different failure zones (distributed into
+physical machines, zones and regions) by observing their labels defined by the standard.
+
+### Errors and warnings
+
+The test will return 0 precisely when it could be verified that the standard is satisfied.
+If no distribution can be detected, a 2 will be returned instead.
+The test mentions if labels are missing, which would hinder the ability to detect node distribution,
+if no distribution is available on specific levels or if not enough nodes are available for
+a distribution to be feasible.
+
+### Implementation
+
+The script [`k8s_node_distribution_check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/kaas/k8s_node_distribution/k8s_node_distribution_check.py)
+connects to an existing K8s cluster and checks if a distribution can be detected with the labels
+set for the nodes of this cluster.
+
+## Manual tests
+
+None.