diff --git a/Standards/scs-0123-v1-mandatory-and-supported-IaaS-services.md b/Standards/scs-0123-v1-mandatory-and-supported-IaaS-services.md index 274738be9..2f7a74326 100644 --- a/Standards/scs-0123-v1-mandatory-and-supported-IaaS-services.md +++ b/Standards/scs-0123-v1-mandatory-and-supported-IaaS-services.md @@ -1,7 +1,8 @@ --- title: Mandatory and Supported IaaS Services type: Standard -status: Draft +status: Stable +stabilized_at: 2024-11-20 track: IaaS --- @@ -40,7 +41,7 @@ The following IaaS APIs MUST be present in SCS-compliant IaaS deployments and co :::caution S3 API implementations may differ in certain offered features. -CSPs must publicly describe, which implementation they use in their deployment. +CSPs must publicly describe the endpoints of their S3 solutions and which implementations they use in their deployment. Users should always research whether a needed feature is supported in the offered implementation. ::: @@ -56,20 +57,19 @@ The following IaaS APIs MAY be present in SCS-compliant IaaS deployment, e.g. im | Supported API | corresponding OpenStack Service | description | |-----|-----|-----| | **bare-metal** | Ironic | Bare Metal provisioning service | -| **billing** | Cloudkitty | Rating/Billing service | +| **billing** | CloudKitty | Rating/Billing service | | **dns** | Designate | DNS service | | **ha** | Masakari | Instances High Availability service | | **key-manager** | Barbican | Key Manager service | | **object-store** | Swift | Object Store with different possible backends | | **orchestration** | Heat | Orchestration service | | **shared-file-systems** | Manila | Shared File Systems service | -| **telemetry** | Ceilometer | Telemetry service | -| **time-series-databse** | Gnocchi | Time Series Database service | +| **time-series-database** | Gnocchi | Time Series Database service | ## Unsupported IaaS APIs All other OpenStack services, whose APIs are not mentioned in the mandatory or supported lists will not be tested for their compatibility and conformance in SCS clouds by the SCS community. -Those services MAY be integrated into IaaS deployments by a Cloud Service Provider on their own responsibility but the SCS will not assume they are present and potential issues that occur during deployment or usage have to be handled by the CSP on their own accord. +Those services MAY be integrated into IaaS deployments by a Cloud Service Provider on their own responsibility but SCS will not assume they are present and potential issues that occur during deployment or usage have to be handled by the CSP on their own accord. The SCS standard offers no guarantees for compatibility or reliability of services categorized as unsupported. ## Related Documents @@ -78,5 +78,5 @@ The SCS standard offers no guarantees for compatibility or reliability of servic ## Conformance Tests -The presence of the mandatory OpenStack APIs will be tested in [this test-script](https://github.com/SovereignCloudStack/standards/blob/mandatory-and-supported-IaaS-services/Tests/iaas/mandatory-services/mandatory-iaas-services.py). -The test will further check, whether the object store endpoint is compatible to s3. +The presence of the mandatory OpenStack APIs will be tested in [this test-script](https://github.com/SovereignCloudStack/standards/blob/main/Tests/iaas/mandatory-services/mandatory-iaas-services.py) +The test will further check whether the object-store endpoint is compatible to s3. diff --git a/Standards/scs-XXXX-v1-security-of-iaas-service-software.md b/Standards/scs-XXXX-v1-security-of-iaas-service-software.md new file mode 100644 index 000000000..94b1200dd --- /dev/null +++ b/Standards/scs-XXXX-v1-security-of-iaas-service-software.md @@ -0,0 +1,138 @@ +--- +title: Standard for the security of IaaS service software +type: Standard +status: Draft +track: IaaS +--- + +## Introduction + +Software security relies on bug patches and security updates being available for specific versions of the software. +The services, which build the IaaS Layer should be updated on a regular basis based on updates provided by their respective authors or distributors. +But older releases or versions of the software of these services may not receive updates anymore. +Unpatched versions should not be used in deployments as they are a security risk, so this standard will define how CSPs should deal with software versions and security updates. + +## Terminology + +| Term | Explanation | +| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| CSP | Cloud Service Provider, provider managing the OpenStack infrastructure. | +| SLURP | Skip Level Upgrade Release Process - A Process that allows upgrades between two releases, while skipping the one in between them. | +| OSSN | [OpenStack Security Note](https://wiki.openstack.org/wiki/Security_Notes) - security issues from 3rd parties or due to misconfigurations. | +| OSSA | [OpenStack Security Advisories](https://security.openstack.org/ossalist.html) - security issues and advices for OpenStack. | + +## Motivation + +On the IaaS Layer the software, that needs to be considered in the scope of this standard, is mainly the APIs of IaaS Services. +Also there might be shared libraries and other dependencies, that could be considered part of the IaaS Layer. +In software projects like e.g. OpenStack that provide the main services and all APIs, the software will be modified and receive bug fixes continuously and will receive releases of new versions on a regular basis. +Older releases will at some point not receive updates anymore, because maintaining more and more releases simultaneously requires too much manpower. +Thus older versions will also eventually not receive security updates anymore. +Using versions which do not receive updates anymore threatens the baseline security of deployments and should be avoided under all circumstances. + +## Design Considerations + +It would be possible to define a minimum version of IaaS Layer software to avoid security risks. +In the following paragraphs several options of defining a minimum version or dealing with security patches otherwise are discussed. + +### Options considered + +#### Only Allow the current versions of Software + +Considering that OpenStack as one provider of IaaS Layer Software has two releases per year, with one SLURP release per year, this option would require CSPs to update their deployment once or twice a year. +Updating a whole deployment is a lot of work and requires also good life-cycle management. +Following only the SLURP releases would reduce this work to once per year. + +While following new releases closely already provides a deployment with recent bug fixes and new features, it also makes developing standards easier. +Differences between releases will accumulate eventually and may render older releases non-compliant to the SCS standards at some point. + +On the other hand on the IaaS Level there aren't many breaking changes introduced by releases and also most standards will also work with older releases. +Security updates and bug fixes are also provided by OpenStack for a few older releases with the state `maintained` according to the OpenStack releases overview[^2]. +Additionally the [SCS reference implementation](https://github.com/SovereignCloudStack/release-notes/blob/main/Release7.md) is integrating OpenStack releases after half a year - so about the time when a new release is published by OpenStack. +Considering a CSP that wants to use only SLURP releases and waits for the reference implementation to adopt them, will already lag over a year (i.e. 2 OpenStack releases) behind the latest release, this cannot be considered as using the current version of IaaS Layer Software. +Thus this option can be discarded. + +#### Allow only maintained versions of Software + +While following closely to the newest releases could be advised, there are several downsides to requiring this workflow, even if it would be only for SLURP releases. +Following the SCS reference implementation for example would also lead into being a little bit behind the newest OpenStack release. +But this is not as bad as it may seem to be, because security related fixes and bug fixes are backported to older but still `maintained` releases. +All releases that are still maintained can be looked up at the releases page from OpenStack[^2]. + +Allowing maintained versions would give CSPs a little bit more time to update and test their environments, while still receiving relevant security updates and bug fixes. +Also CSPs that want to become SCS-compliant will not have to take on the burden to upgrade their deployments to very recent releases immediately, but can instead test with an existing release before an upgrade and identify where they need to put in additional work to become SCS-compliant. + +One problem is, that there might be new features implemented in the newest versions of the software, which are desired by other SCS standards to be SCS-compliant. +In that case allowing all maintained versions would lead to a two-year timespan customers would need to wait for before such a feature becomes available in all SCS-compliant deployments. +In case of security relevant features this is not advisable. + +#### Standards implicitly define the minimum versions of Software + +Instead of requiring a defined minimum software version centrally, it could be derived from the individual standards. +Because: Whenever there is a new wanted behavior a standard should be created and a resonable timeframe given to CSPs to adopt a software version that can fulfill the new standard. +Through the combination of all standards that are in place, the minimum version for the IaaS service software is implicitly given. + +This would avoid to have conflicting versions of software in terms of feature parity, while also allowing older software. +Using this approach requires an additional advise to CSPs to update or implement patches for security issues. + +#### Advise CSPs to integrate software updates + +As long as maintained versions of software are used, updates with security patches are available and only need to be integrated. +This can and should be done in a reasonable short timeframe. + +But CSPs may even use releases of IaaS software, that are either not maintained anymore by an open source community or may be even closed source implementations of the mandatory IaaS APIs. +Allowing older versions or closed source software would only be acceptable, when CSPs assure (e.g. in documentation), that they themself will patch the software within their deployments. +Security bug fixes must be implemented and proof of the fix then provided. +Only under these circumstances deployments with older or alternative IaaS Layer software may be handled as compliant. + +This option could be taken for granted, but to actually advise using it may encourage CSPs to take a closer look on their life-cycle management and security risk handling. +And CSPs using OpenStack could even be encouraged to upgrade their deployments. + +#### Dependencies of the IaaS Layer Software + +While the IaaS service software like OpenStack itself is monitored and security issues announced in OSSNs and OSSAs, these services have lots of dependecies, that are not monitored by the same entity. +When dependencies have security issues, there might be no OSSN or OSSA, so CSPs also need to watch CVEs concerning these dependencies themselves. +Those dependencies must also be updated in a reasonable timeframe, when a security issue is disclosed. + +#### What timeframe is needed to fix the issue? + +CSPs should be encouraged to fix security issues as fast as possible. +Some security issues are very easy to exploit so as soon as the vulnerability is disclosed attacks on deployments will start. +Other vulnerabilities may need much knowledge and more time to be exploited. +Also the impact of different vulnerabilities will differ. + +So it can be concluded that some security issues need to be fixed immediately while for others it is okay to take some time. +The BSI already has some guidance[^1] on how fast CSPs should respond. +From the moment a vulnerability is disclosed these are the advised reaction times ranked by the severity of the vulnerability: + +1. Critical (CVSS = 9.0 – 10.0): 3 hours +2. High (CVSS = 7.0 – 8.9): 3 days +3. Mid (CVSS = 4.0 – 6.9): 1 month +4. Low (CVSS = 0.1 – 3.9): 3 months + +[^1]: [C5 criteria catalog with timeframes for responses on page 70.](https://www.bsi.bund.de/SharedDocs/Downloads/EN/BSI/CloudComputing/ComplianceControlsCatalogue/2020/C5_2020.pdf?__blob=publicationFile&v=3) + +This standard will follow this guidance and refer to these timeframes as "reasonable timeframes". + +## Standard for a minimum IaaS Layer Software version + +If a deployment is affected by a security issue and a maintained[^2] version of OpenStack is used as implementation for IaaS Layer software, security patches noted in OSSNs and OSSAs MUST be integrated within a reasonable timeframe according to the severity of the security issue[^1]. +Otherwise the CSP MUST implement security bug fixes themself within a reasonable timeframe, when the deplyoment is affected by a security issue according to the severity of the security issue[^1]. + +In both cases a notice of the update MUST be send to the OSBA, so that the compliance will not be revoked. + +If a deployment uses a dependency of the IaaS service software which is affected by a security issue, this software also MUST be updated with security patches within a reasonable timeframe[^1]. + +An open SBOM list MAY be used to propagate the current version of the software and may be used as proof of updates. + +[^2]: [OpenStack versions and their current status](https://releases.openstack.org) + +## Conformance Tests + +In case of provided SBOMs the version numbers of the software could be checked. +But this is not a requirement, so there cannot be such a test. +Tests on the integration of security patches itself are difficult. +And even if tests for certain security issues are possible, then those might be interpreted as an attack. +This is the reason there will be no conformance test. + +Rather the standard requires that CSPs provide notice of the fixed vulnerabilites themselves. diff --git a/Standards/scs-XXXX-w1-security-of-iaas-service-software.md b/Standards/scs-XXXX-w1-security-of-iaas-service-software.md new file mode 100644 index 000000000..3f0b1df8c --- /dev/null +++ b/Standards/scs-XXXX-w1-security-of-iaas-service-software.md @@ -0,0 +1,45 @@ +--- +title: "SCS Standard for the security of IaaS service software: Implementation and Testing Notes" +type: Supplement +track: IaaS +status: Draft +supplements: + - scs-XXXX-v1-security-of-iaas-service-software.md +--- + +## Testing or Detecting security updates in software + +It is not always possible to automatically test, whether the software has the newest security updates. +This is because software versions may differ or some CSPs might have added downstream code parts or using other software than the reference. +Also vulnerabilites and their fixes are quite different in testing, some might not be testable while others are. +Additionally testing might be perceived as an attack on the infrastructure. +So this standard will rely on the work and information CSPs must provide. +There are different cases and procedures which are addressed in the following parts, that lead to compliance for this standard. + +### Procedure to become compliant to the security of IaaS service software Standard + +This is the procedure when a new deployment wants to achieve SCS-conformancy. +There are two states such a deployment can be in: + +1. When a deployment is newly build or installed it usually uses software which includes all the latest security and bug fixes. +Such deployments should be considered compliant to the standard. + +2. When a CSP wants to make an older deployment compliant to the SCS standards and thus also to this standard, it should be checked, whether the running software is up to date and all vulnerabilites are fixed. +Any updates or upgrades to even newer versions should be done before the SCS compliance for every other standard is checked. +Afterwards the CSP may provide information about the used software in an SBOM or otherwise should provide a notice about the deployment having integrated all necessary vulnerability patches. + +### Procedure when new vulnerabilites are discovered + +Whenever there are new vulnerabilities discovered in IaaS service software like OpenStack there is either an internal discussion ongoing or it is just a smaller issue. +In the first case CSPs should have someone following such discussions and may even help preparing and testing patches. +From the moment on the vulnerability is disclosed publicly, the risk of it being actively exploited increases greatly. +So CSPs MUST watch out for announcements like in the OSSAs and OSSNs and when they are affected, update their deployment within the following timeframes according to the severity of the issue: + +1. Critical (CVSS = 9.0 – 10.0): 3 hours +2. High (CVSS = 7.0 – 8.9): 3 days +3. Mid (CVSS = 4.0 – 6.9): 1 month +4. Low (CVSS = 0.1 – 3.9): 3 months + +Afterwards CSPs MUST provide a notice to the OSBA, that they are not or not anymore affected by the vulnerabilty. +This can be done through either telling, what patches were integrated or showing configuration that renders the attack impossible. +It could also be provided a list of services, when the affected service is not used in that deployment. diff --git a/Tests/config.toml b/Tests/config.toml index a0173c25d..6b7a5c71f 100644 --- a/Tests/config.toml +++ b/Tests/config.toml @@ -34,11 +34,51 @@ subjects = [ workers = 4 +[presets.kaas-dev] +scopes = [ + "scs-compatible-kaas", +] +subjects = [ + "kind-current", + "kind-current-1", + "kind-current-2", +] +workers = 1 # better restrict this with clusters running on local machine + + [scopes.scs-compatible-iaas] spec = "./scs-compatible-iaas.yaml" +[scopes.scs-compatible-kaas] +spec = "./scs-compatible-kaas.yaml" + + # default subject (not a real subject, but used to declare a default mapping) # (this is the only mapping declaration that supports using Python string interpolation) [subjects._.mapping] os_cloud = "{subject}" +subject_root = "{subject}" + + +[subjects._.kubernetes_setup] +clusterspec = "kaas/clusterspec.yaml" + + +[subjects.kind-current.kubernetes_setup] +kube_plugin = "kind" +kube_plugin_config = "kaas/kind_config.yaml" +clusterspec_cluster = "current-k8s-release" + + +[subjects.kind-current-1.kubernetes_setup] +kube_plugin = "kind" +kube_plugin_config = "kaas/kind_config.yaml" +clusterspec_cluster = "current-k8s-release-1" + + +[subjects.kind-current-2.kubernetes_setup] +kube_plugin = "kind" +kube_plugin_config = "kaas/kind_config.yaml" +clusterspec_cluster = "current-k8s-release-2" + diff --git a/Tests/kaas/clusterspec.yaml b/Tests/kaas/clusterspec.yaml new file mode 100644 index 000000000..c8439a89f --- /dev/null +++ b/Tests/kaas/clusterspec.yaml @@ -0,0 +1,11 @@ +# this file specifies all clusters that have to be provisioned for the tests to run +clusters: + current-k8s-release: + branch: "1.31" + kubeconfig: kubeconfig.yaml + current-k8s-release-1: + branch: "1.30" + kubeconfig: kubeconfig.yaml + current-k8s-release-2: + branch: "1.29" + kubeconfig: kubeconfig.yaml diff --git a/Tests/kaas/k8s-version-policy/k8s-eol-data.yml b/Tests/kaas/k8s-version-policy/k8s-eol-data.yml index 6a549c464..3a3d3b2eb 100644 --- a/Tests/kaas/k8s-version-policy/k8s-eol-data.yml +++ b/Tests/kaas/k8s-version-policy/k8s-eol-data.yml @@ -1,5 +1,7 @@ # https://kubernetes.io/releases/patch-releases/#detailed-release-history-for-active-branches +- branch: '1.31' + end-of-life: '2025-10-28' - branch: '1.30' end-of-life: '2025-06-28' - branch: '1.29' diff --git a/Tests/kaas/kind_config.yaml b/Tests/kaas/kind_config.yaml new file mode 100644 index 000000000..ead21eb72 --- /dev/null +++ b/Tests/kaas/kind_config.yaml @@ -0,0 +1,5 @@ +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +nodes: +- role: control-plane +- role: worker diff --git a/Tests/kaas/plugin/README.md b/Tests/kaas/plugin/README.md new file mode 100644 index 000000000..e54cf1864 --- /dev/null +++ b/Tests/kaas/plugin/README.md @@ -0,0 +1,38 @@ +# Plugin for provisioning k8s clusters and performing conformance tests on these clusters + +## Development environment + +### requirements + +* [docker](https://docs.docker.com/engine/install/) +* [kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation) + +### setup for development + +1. Generate python 3.10 env + + ```bash + sudo apt-get install python3.10-dev + virtualenv -p /usr/bin/python3.10 venv + echo "*" >> venv/.gitignore + source venv/bin/activate + (venv) curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10 + (venv) python3.10 -m pip install --upgrade pip + (venv) python3.10 -m pip --version + + ``` + +2. Install dependencies: + + ```bash + (venv) pip install pip-tools + (venv) pip-compile requirements.in + (venv) pip-sync requirements.txt + ``` + +3. Set environment variables and launch the process: + + ```bash + (venv) export CLUSTER_PROVIDER="kind" + (venv) python run.py + ``` diff --git a/Tests/kaas/plugin/interface.py b/Tests/kaas/plugin/interface.py new file mode 100644 index 000000000..f62e3b3e2 --- /dev/null +++ b/Tests/kaas/plugin/interface.py @@ -0,0 +1,54 @@ + + +class KubernetesClusterPlugin(): + """ + An abstract base class for custom Kubernetes cluster provider plugins. + It represents an interface class from which the api provider-specific + plugins must be derived as child classes + + To implement fill the methods `create_cluster` and `delete_cluster` with + api provider-specific functionalities for creating and deleting clusters. + The `create_cluster` method must ensure that the kubeconfigfile is provided + at the position in the file system defined by the parameter + `kubeconfig_filepath` + + - Implement `create_cluster` and `delete_cluster` methods + - Create `__init__(self, config_file)` method to handle api specific + configurations. + + Example: + .. code:: python + + from interface import KubernetesClusterPlugin + from apiX_library import cluster_api_class as ClusterAPI + + class PluginX(KubernetesClusterPlugin): + + def __init__(self, config_file): + self.config = config_file + + def create_cluster(self, cluster_name, version, kubeconfig_filepath): + self.cluster = ClusterAPI(name=cluster_name, image=cluster_image, kubeconfig_filepath) + self.cluster.create(self.config) + + def delete_cluster(self, cluster_name): + self.cluster = ClusterAPI(cluster_name) + self.cluster.delete() + .. + """ + + def create_cluster(self, cluster_name, version, kubeconfig_filepath): + """ + This method is to be called to create a k8s cluster + :param: cluster_name: + :param: version: + :param: kubeconfig_filepath: + """ + raise NotImplementedError + + def delete_cluster(self, cluster_name): + """ + This method is to be called in order to unprovision a cluster + :param: cluster_name: + """ + raise NotImplementedError diff --git a/Tests/kaas/plugin/plugin_kind.py b/Tests/kaas/plugin/plugin_kind.py new file mode 100644 index 000000000..26cd3f23d --- /dev/null +++ b/Tests/kaas/plugin/plugin_kind.py @@ -0,0 +1,50 @@ +import logging +import os +import os.path +from pathlib import Path + +from interface import KubernetesClusterPlugin +from pytest_kind import KindCluster + +logger = logging.getLogger(__name__) + + +class PluginKind(KubernetesClusterPlugin): + """ + Plugin to handle the provisioning of kubernetes cluster for + conformance testing purpose with the use of Kind + """ + def __init__(self, config_path): + logger.info("Init PluginKind") + self.config = config_path + logger.debug(self.config) + self.working_directory = os.getcwd() + logger.debug(f"Working from {self.working_directory}") + + def create_cluster(self, cluster_name, version, kubeconfig): + """ + This method is to be called to create a k8s cluster + :param: kubernetes_version: + :return: kubeconfig_filepath + """ + cluster_version = version + if cluster_version == '1.29': + cluster_version = 'v1.29.8' + elif cluster_version == '1.30': + cluster_version = 'v1.30.4' + elif cluster_version == '1.31' or cluster_version == 'default': + cluster_version = 'v1.31.1' + cluster_image = f"kindest/node:{cluster_version}" + kubeconfig_filepath = Path(kubeconfig) + if kubeconfig_filepath is None: + raise ValueError("kubeconfig_filepath is missing") + else: + self.cluster = KindCluster(name=cluster_name, image=cluster_image, kubeconfig=kubeconfig_filepath) + if self.config is None: + self.cluster.create() + else: + self.cluster.create(self.config) + + def delete_cluster(self, cluster_name): + self.cluster = KindCluster(cluster_name) + self.cluster.delete() diff --git a/Tests/kaas/plugin/plugin_static.py b/Tests/kaas/plugin/plugin_static.py new file mode 100644 index 000000000..0bd24707e --- /dev/null +++ b/Tests/kaas/plugin/plugin_static.py @@ -0,0 +1,19 @@ +import shutil + +from interface import KubernetesClusterPlugin + + +class PluginStatic(KubernetesClusterPlugin): + """ + Plugin to handle the provisioning of kubernetes + using a kubeconfig file + """ + + def __init__(self, config_path): + self.kubeconfig_path = config_path + + def create_cluster(self, cluster_name, version, kubeconfig): + shutil.copyfile(self.kubeconfig_path, kubeconfig) + + def delete_cluster(self, cluster_name, version): + pass diff --git a/Tests/kaas/plugin/requirements.in b/Tests/kaas/plugin/requirements.in new file mode 100644 index 000000000..0a60c3c3c --- /dev/null +++ b/Tests/kaas/plugin/requirements.in @@ -0,0 +1,2 @@ +pytest-kind +kubernetes diff --git a/Tests/kaas/plugin/requirements.txt b/Tests/kaas/plugin/requirements.txt new file mode 100644 index 000000000..a04a03167 --- /dev/null +++ b/Tests/kaas/plugin/requirements.txt @@ -0,0 +1,60 @@ +# +# This file is autogenerated by pip-compile with Python 3.12 +# by the following command: +# +# pip-compile requirements.in +# +cachetools==5.5.0 + # via google-auth +certifi==2024.8.30 + # via + # kubernetes + # requests +charset-normalizer==3.3.2 + # via requests +google-auth==2.34.0 + # via kubernetes +idna==3.8 + # via requests +kubernetes==30.1.0 + # via -r requirements.in +oauthlib==3.2.2 + # via + # kubernetes + # requests-oauthlib +pyasn1==0.6.0 + # via + # pyasn1-modules + # rsa +pyasn1-modules==0.4.0 + # via google-auth +pykube-ng==23.6.0 + # via pytest-kind +pytest-kind==22.11.1 + # via -r requirements.in +python-dateutil==2.9.0.post0 + # via kubernetes +pyyaml==6.0.2 + # via + # kubernetes + # pykube-ng +requests==2.32.3 + # via + # kubernetes + # pykube-ng + # requests-oauthlib +requests-oauthlib==2.0.0 + # via kubernetes +rsa==4.9 + # via google-auth +six==1.16.0 + # via + # kubernetes + # python-dateutil +urllib3==2.2.2 + # via + # kubernetes + # pykube-ng + # requests +websocket-client==1.8.0 + # via kubernetes diff --git a/Tests/kaas/plugin/run_plugin.py b/Tests/kaas/plugin/run_plugin.py new file mode 100755 index 000000000..7b4084107 --- /dev/null +++ b/Tests/kaas/plugin/run_plugin.py @@ -0,0 +1,58 @@ +#!/usr/bin/env python3 +import logging +import os.path + +import click +import yaml + +from plugin_kind import PluginKind +from plugin_static import PluginStatic + +PLUGIN_LOOKUP = { + "kind": PluginKind, + "static": PluginStatic, +} + + +def init_plugin(plugin_kind, config_path): + plugin_maker = PLUGIN_LOOKUP.get(plugin_kind) + if plugin_maker is None: + raise ValueError(f"unknown plugin '{plugin_kind}'") + return plugin_maker(config_path) + + +def load_spec(clusterspec_path): + with open(clusterspec_path, "rb") as fileobj: + return yaml.load(fileobj, Loader=yaml.SafeLoader) + + +@click.group() +def cli(): + pass + + +@cli.command() +@click.argument('plugin_kind', type=click.Choice(list(PLUGIN_LOOKUP), case_sensitive=False)) +@click.argument('plugin_config', type=click.Path(exists=True, dir_okay=False)) +@click.argument('clusterspec_path', type=click.Path(exists=True, dir_okay=False)) +@click.argument('cluster_id', type=str, default="default") +def create(plugin_kind, plugin_config, clusterspec_path, cluster_id): + clusterspec = load_spec(clusterspec_path)['clusters'] + plugin = init_plugin(plugin_kind, plugin_config) + clusterinfo = clusterspec[cluster_id] + plugin.create_cluster(cluster_id, clusterinfo['branch'], os.path.abspath(clusterinfo['kubeconfig'])) + + +@cli.command() +@click.argument('plugin_kind', type=click.Choice(list(PLUGIN_LOOKUP), case_sensitive=False)) +@click.argument('plugin_config', type=click.Path(exists=True, dir_okay=False)) +@click.argument('clusterspec_path', type=click.Path(exists=True, dir_okay=False)) +@click.argument('cluster_id', type=str, default="default") +def delete(plugin_kind, plugin_config, clusterspec_path, cluster_id): + plugin = init_plugin(plugin_kind, plugin_config) + plugin.delete_cluster(cluster_id) + + +if __name__ == '__main__': + logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.INFO) + cli() diff --git a/Tests/requirements.txt b/Tests/requirements.txt index 9505a7061..bf93aff83 100644 --- a/Tests/requirements.txt +++ b/Tests/requirements.txt @@ -6,7 +6,7 @@ # aiohappyeyeballs==2.3.5 # via aiohttp -aiohttp==3.10.3 +aiohttp==3.10.11 # via # -r requirements.in # kubernetes-asyncio @@ -76,6 +76,8 @@ pbr==6.0.0 # stevedore platformdirs==4.2.2 # via openstacksdk +propcache==0.2.0 + # via yarl pycparser==2.22 # via cffi python-dateutil==2.9.0.post0 @@ -109,5 +111,5 @@ urllib3==2.2.2 # via # kubernetes-asyncio # requests -yarl==1.9.4 +yarl==1.17.2 # via aiohttp diff --git a/Tests/scs-compatible-kaas.yaml b/Tests/scs-compatible-kaas.yaml index 4aa540999..a4010c64e 100644 --- a/Tests/scs-compatible-kaas.yaml +++ b/Tests/scs-compatible-kaas.yaml @@ -2,7 +2,9 @@ name: SCS-compatible KaaS uuid: 1fffebe6-fd4b-44d3-a36c-fc58b4bb0180 url: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Tests/scs-compatible-kaas.yaml variables: - - kubeconfig + - subject_root + # directory containing the kubeconfig file for the subject under test + # (note that we consider each kubernetes branch a test subject of its own) modules: - id: cncf-k8s-conformance name: CNCF Kubernetes conformance @@ -12,38 +14,30 @@ modules: tags: [mandatory] - id: scs-0210-v2 name: Kubernetes version policy - url: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0210-v2-k8s-version-policy.md + url: https://docs.scs.community/standards/scs-0210-v2-k8s-version-policy run: - executable: ./kaas/k8s-version-policy/k8s_version_policy.py - args: -k {kubeconfig} + args: -k {subject_root}/kubeconfig.yaml testcases: - id: version-policy-check tags: [mandatory] - id: scs-0214-v2 name: Kubernetes node distribution and availability - url: https://raw.githubusercontent.com/SovereignCloudStack/standards/main/Standards/scs-0214-v1-k8s-node-distribution.md + url: https://docs.scs.community/standards/scs-0214-v2-k8s-node-distribution run: - executable: ./kaas/k8s-node-distribution/k8s_node_distribution_check.py - args: -k {kubeconfig} + args: -k {subject_root}/kubeconfig.yaml testcases: - id: node-distribution-check tags: [mandatory] timeline: - # empty timeline might confuse tools, so put one "dummy" entry here - date: 2024-02-28 versions: v1: draft - v2: draft versions: - - version: v2 - include: - - cncf-k8s-conformance - - scs-0210-v2 - - scs-0214-v2 - targets: - main: mandatory - version: v1 include: + - cncf-k8s-conformance - scs-0210-v2 - scs-0214-v2 targets: diff --git a/Tests/scs-test-runner.py b/Tests/scs-test-runner.py index de7152428..780601e96 100755 --- a/Tests/scs-test-runner.py +++ b/Tests/scs-test-runner.py @@ -17,16 +17,23 @@ import click import tomli - logger = logging.getLogger(__name__) MONITOR_URL = "https://compliance.sovereignit.cloud/" +def ensure_dir(path): + try: + os.makedirs(path) + except FileExistsError: + pass + + class Config: def __init__(self): self.cwd = os.path.abspath(os.path.dirname(sys.argv[0]) or os.getcwd()) self.scs_compliance_check = os.path.join(self.cwd, 'scs-compliance-check.py') self.cleanup_py = os.path.join(self.cwd, 'cleanup.py') + self.run_plugin_py = os.path.join(self.cwd, 'kaas', 'plugin', 'run_plugin.py') self.ssh_keygen = shutil.which('ssh-keygen') self.curl = shutil.which('curl') self.secrets = {} @@ -58,42 +65,80 @@ def get_subject_mapping(self, subject): mapping.update(self.subjects.get(subject, {}).get('mapping', {})) return mapping + def get_kubernetes_setup(self, subject): + default_kubernetes_setup = self.subjects.get('_', {}).get('kubernetes_setup', {}) + kubernetes_setup = dict(default_kubernetes_setup) + kubernetes_setup.update(self.subjects.get(subject, {}).get('kubernetes_setup', {})) + return kubernetes_setup + def abspath(self, path): return os.path.join(self.cwd, path) def build_check_command(self, scope, subject, output): # TODO figure out when to supply --debug here (but keep separated from our --debug) - cmd = [ + args = [ sys.executable, self.scs_compliance_check, self.abspath(self.scopes[scope]['spec']), '--debug', '-C', '-o', output, '-s', subject, ] for key, value in self.get_subject_mapping(subject).items(): - cmd.extend(['-a', f'{key}={value}']) - return cmd + args.extend(['-a', f'{key}={value}']) + return {'args': args} + + def build_provision_command(self, subject): + kubernetes_setup = self.get_kubernetes_setup(subject) + subject_root = self.abspath(self.get_subject_mapping(subject).get('subject_root') or '.') + ensure_dir(subject_root) + return { + 'args': [ + sys.executable, self.run_plugin_py, + 'create', + kubernetes_setup['kube_plugin'], + self.abspath(kubernetes_setup['kube_plugin_config']), + self.abspath(kubernetes_setup['clusterspec']), + kubernetes_setup['clusterspec_cluster'], + ], + 'cwd': subject_root, + } + + def build_unprovision_command(self, subject): + kubernetes_setup = self.get_kubernetes_setup(subject) + subject_root = self.abspath(self.get_subject_mapping(subject).get('subject_root') or '.') + ensure_dir(subject_root) + return { + 'args': [ + sys.executable, self.run_plugin_py, + 'delete', + kubernetes_setup['kube_plugin'], + self.abspath(kubernetes_setup['kube_plugin_config']), + self.abspath(kubernetes_setup['clusterspec']), + kubernetes_setup['clusterspec_cluster'], + ], + 'cwd': subject_root, + } def build_cleanup_command(self, subject): # TODO figure out when to supply --debug here (but keep separated from our --debug) - return [ + return {'args': [ sys.executable, self.cleanup_py, '-c', self.get_subject_mapping(subject)['os_cloud'], '--prefix', '_scs-', '--ipaddr', '10.1.0.', '--debug', - ] + ]} def build_sign_command(self, target_path): - return [ + return {'args': [ self.ssh_keygen, '-Y', 'sign', '-f', self.abspath(self.secrets['keyfile']), '-n', 'report', target_path, - ] + ]} def build_upload_command(self, target_path, monitor_url): if not monitor_url.endswith('/'): monitor_url += '/' - return [ + return {'args': [ self.curl, '--fail-with-body', '--data-binary', f'@{target_path}.sig', @@ -101,7 +146,7 @@ def build_upload_command(self, target_path, monitor_url): '-H', 'Content-Type: application/x-signed-yaml', '-H', f'Authorization: Basic {self.auth_token}', f'{monitor_url}reports', - ] + ]} @click.group() @@ -123,7 +168,7 @@ def _run_commands(commands, num_workers=5): processes = [] while commands or processes: while commands and len(processes) < num_workers: - processes.append(subprocess.Popen(commands.pop())) + processes.append(subprocess.Popen(**commands.pop())) processes[:] = [p for p in processes if p.poll() is None] time.sleep(0.5) @@ -180,22 +225,14 @@ def run(cfg, scopes, subjects, preset, num_workers, monitor_url, report_yaml): commands = [cfg.build_check_command(job[0], job[1], output) for job, output in zip(jobs, outputs)] _run_commands(commands, num_workers=num_workers) _concat_files(outputs, report_yaml_tmp) - subprocess.run(cfg.build_sign_command(report_yaml_tmp)) - subprocess.run(cfg.build_upload_command(report_yaml_tmp, monitor_url)) + subprocess.run(**cfg.build_sign_command(report_yaml_tmp)) + subprocess.run(**cfg.build_upload_command(report_yaml_tmp, monitor_url)) if report_yaml is not None: _move_file(report_yaml_tmp, report_yaml) return 0 -@cli.command() -@click.option('--subject', 'subjects', type=str) -@click.option('--preset', 'preset', type=str) -@click.option('--num-workers', 'num_workers', type=int, default=5) -@click.pass_obj -def cleanup(cfg, subjects, preset, num_workers): - """ - clean up any lingering resources - """ +def _run_command_for_subjects(cfg, subjects, preset, num_workers, command): if not subjects and not preset: preset = 'default' if preset: @@ -208,12 +245,49 @@ def cleanup(cfg, subjects, preset, num_workers): subjects = [subject.strip() for subject in subjects.split(',')] if subjects else [] if not subjects: raise click.UsageError('subject(s) must be non-empty') - logger.debug(f'cleaning up for subject(s) {", ".join(subjects)}, num_workers: {num_workers}') - commands = [cfg.build_cleanup_command(subject) for subject in subjects] + logger.debug(f'running {command} for subject(s) {", ".join(subjects)}, num_workers: {num_workers}') + m = getattr(cfg, f'build_{command}_command') + commands = [m(subject) for subject in subjects] _run_commands(commands, num_workers=num_workers) return 0 +@cli.command() +@click.option('--subject', 'subjects', type=str) +@click.option('--preset', 'preset', type=str) +@click.option('--num-workers', 'num_workers', type=int, default=5) +@click.pass_obj +def cleanup(cfg, subjects, preset, num_workers): + """ + clean up any lingering IaaS resources + """ + return _run_command_for_subjects(cfg, subjects, preset, num_workers, "cleanup") + + +@cli.command() +@click.option('--subject', 'subjects', type=str) +@click.option('--preset', 'preset', type=str) +@click.option('--num-workers', 'num_workers', type=int, default=5) +@click.pass_obj +def provision(cfg, subjects, preset, num_workers): + """ + create k8s clusters + """ + return _run_command_for_subjects(cfg, subjects, preset, num_workers, "provision") + + +@cli.command() +@click.option('--subject', 'subjects', type=str) +@click.option('--preset', 'preset', type=str) +@click.option('--num-workers', 'num_workers', type=int, default=5) +@click.pass_obj +def unprovision(cfg, subjects, preset, num_workers): + """ + clean up k8s clusters + """ + return _run_command_for_subjects(cfg, subjects, preset, num_workers, "unprovision") + + if __name__ == '__main__': logging.basicConfig(format='%(levelname)s: %(message)s', level=logging.INFO) cli(obj=Config()) diff --git a/package-lock.json b/package-lock.json index e20b3925b..1fcf557e6 100644 --- a/package-lock.json +++ b/package-lock.json @@ -152,9 +152,9 @@ } }, "node_modules/cross-spawn": { - "version": "7.0.3", - "resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.3.tgz", - "integrity": "sha512-iRDPJKUPVEND7dHPO8rkbOnPpyDygcDFtWjpeWNCgy8WP2rXcxXL8TskReQl6OrB2G7+UJrags1q15Fudc7G6w==", + "version": "7.0.6", + "resolved": "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz", + "integrity": "sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA==", "dependencies": { "path-key": "^3.1.0", "shebang-command": "^2.0.0",