Skip to content

Commit

Permalink
Merge branch 'main' into 526-refine-cve-check-in-scs-0210-v2-test-script
Browse files Browse the repository at this point in the history
  • Loading branch information
piobig2871 authored Nov 28, 2024
2 parents f51a46c + 53b5e45 commit 48ba4d7
Show file tree
Hide file tree
Showing 27 changed files with 1,309 additions and 133 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ type: Supplement
track: IaaS
status: Draft
supplements:
- scs-XXXX-v1-security-of-iaas-service-software.md
- scs-0124-v1-security-of-iaas-service-software.md
---

## Testing or Detecting security updates in software
Expand Down
277 changes: 277 additions & 0 deletions Standards/scs-0125-v1-secure-connections.md

Large diffs are not rendered by default.

1 change: 0 additions & 1 deletion Standards/scs-0214-v1-k8s-node-distribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,4 +84,3 @@ If the standard is used by a provider, the following decisions are binding and v
[k8s-ha]: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/
[k8s-large-clusters]: https://kubernetes.io/docs/setup/best-practices/cluster-large/
[scs-0213-v1]: https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0213-v1-k8s-nodes-anti-affinity.md
[k8s-labels-docs]: https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone
20 changes: 2 additions & 18 deletions Standards/scs-0214-v2-k8s-node-distribution.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
title: Kubernetes Node Distribution and Availability
type: Standard
status: Draft
status: Stable
stabilized_at: 2024-11-21
replaces: scs-0214-v1-k8s-node-distribution.md
track: KaaS
---
Expand Down Expand Up @@ -100,23 +101,6 @@ These labels MUST be kept up to date with the current state of the deployment.
The field gets autopopulated most of the time by either the kubelet or external mechanisms
like the cloud controller.

- `topology.scs.community/host-id`

This is an SCS-specific label; it MUST contain the hostID of the physical machine running
the hypervisor (NOT: the hostID of a virtual machine). Here, the hostID is an arbitrary identifier,
which need not contain the actual hostname, but it should nonetheless be unique to the host.
This helps identify the distribution over underlying physical machines,
which would be masked if VM hostIDs were used.

## Conformance Tests

The script `k8s-node-distribution-check.py` checks the nodes available with a user-provided
kubeconfig file. Based on the labels `topology.scs.community/host-id`,
`topology.kubernetes.io/zone`, `topology.kubernetes.io/region` and `node-role.kubernetes.io/control-plane`,
the script then determines whether the nodes are distributed according to this standard.
If this isn't the case, the script produces an error.
It also produces warnings and informational outputs, e.g., if labels don't seem to be set.

## Previous standard versions

This is version 2 of the standard; it extends [version 1](scs-0214-v1-k8s-node-distribution.md) with the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,25 +16,15 @@ Worker nodes can also be distributed over "failure zones", but this isn't a requ
Distribution must be shown through labelling, so that users can access these information.

Node distribution metadata is provided through the usage of the labels
`topology.kubernetes.io/region`, `topology.kubernetes.io/zone` and
`topology.scs.community/host-id` respectively.

At the moment, not all labels are set automatically by most K8s cluster utilities, which incurs
additional setup and maintenance costs.
`topology.kubernetes.io/region` and `topology.kubernetes.io/zone`.

## Automated tests

### Notes

The test for the [SCS K8s Node Distribution and Availability](https://github.com/SovereignCloudStack/standards/blob/main/Standards/scs-0214-v2-k8s-node-distribution.md)
checks if control-plane nodes are distributed over different failure zones (distributed into
physical machines, zones and regions) by observing their labels defined by the standard.

### Implementation
Currently, automated testing is not readily possible because we cannot access information about
the underlying host of a node (as opposed to its region and zone). Therefore, the test will only output
a tentative result.

The script [`k8s_node_distribution_check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/kaas/k8s-node-distribution/k8s_node_distribution_check.py)
connects to an existing K8s cluster and checks if a distribution can be detected with the labels
set for the nodes of this cluster.
The current implementation can be found in the script [`k8s_node_distribution_check.py`](https://github.com/SovereignCloudStack/standards/blob/main/Tests/kaas/k8s-node-distribution/k8s_node_distribution_check.py).

## Manual tests

Expand Down
3 changes: 2 additions & 1 deletion Standards/scs-0219-v1-kaas-networking.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
---
title: KaaS Networking Standard
type: Standard
status: Draft
status: Stable
stabilized_at: 2024-11-21
track: KaaS
---

Expand Down
1 change: 1 addition & 0 deletions Tests/.gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
htmlcov/
.coverage
.secret
90 changes: 90 additions & 0 deletions Tests/add_subject.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
#!/usr/bin/env python3
# vim: set ts=4 sw=4 et:
#
# add_subject.py
#
# (c) Matthias Büchse <[email protected]>
# SPDX-License-Identifier: Apache-2.0
import base64
import getpass
import os
import os.path
import re
import shutil
import signal
import subprocess
import sys

try:
from passlib.context import CryptContext
import argon2 # noqa:F401
except ImportError:
print('Missing passlib and/or argon2. Please do:\npip install passlib argon2_cffi', file=sys.stderr)
sys.exit(1)

# see ../compliance-monitor/monitor.py
CRYPTCTX = CryptContext(schemes=('argon2', 'bcrypt'), deprecated='auto')
SSH_KEYGEN = shutil.which('ssh-keygen')
SUBJECT_RE = re.compile(r"[a-zA-Z0-9_\-]+")


def main(argv, cwd):
if len(argv) != 1:
raise RuntimeError("Need to supply precisely one argument: name of subject")
subject = argv[0]
print(f"Attempt to add subject {subject!r}")
keyfile_path = os.path.join(cwd, '.secret', 'keyfile')
tokenfile_path = os.path.join(cwd, '.secret', 'tokenfile')
if os.path.exists(keyfile_path):
raise RuntimeError(f"Keyfile {keyfile_path} already present. Please proceed manually")
if os.path.exists(tokenfile_path):
raise RuntimeError(f"Tokenfile {tokenfile_path} already present. Please proceed manually")
if not SUBJECT_RE.fullmatch(subject):
raise RuntimeError(f"Subject name {subject!r} using disallowed characters")
sanitized_subject = subject.replace('-', '_')
print("Creating API key...")
while True:
password = getpass.getpass("Enter passphrase: ")
if password == getpass.getpass("Repeat passphrase: "):
break
print("No match. Try again...")
token = base64.b64encode(f"{subject}:{password}".encode('utf-8'))
hash_ = CRYPTCTX.hash(password)
with open(tokenfile_path, "wb") as fileobj:
os.fchmod(fileobj.fileno(), 0o600)
fileobj.write(token)
print("Creating key file using `ssh-keygen`...")
subprocess.check_call([SSH_KEYGEN, '-t', 'ed25519', '-C', sanitized_subject, '-f', keyfile_path, '-N', '', '-q'])
with open(keyfile_path + '.pub', "r") as fileobj:
pubkey_components = fileobj.readline().split()
print(f'''
The following SECRET files have been created:
- {keyfile_path}
- {tokenfile_path}
They are required for submitting test reports. You MUST keep them secure and safe.
Insert the following snippet into compliance-monitor/bootstrap.yaml:
- subject: {subject}
api_keys:
- "{hash_}"
keys:
- public_key: "{pubkey_components[1]}"
public_key_type: "{pubkey_components[0]}"
public_key_name: "primary"
Make sure to submit a pull request with the changed file. Otherwise, the reports cannot be submitted.
''', end='')


if __name__ == "__main__":
try:
sys.exit(main(sys.argv[1:], cwd=os.path.dirname(sys.argv[0]) or os.getcwd()) or 0)
except RuntimeError as e:
print(str(e), file=sys.stderr)
sys.exit(1)
except KeyboardInterrupt:
print("Interrupted", file=sys.stderr)
sys.exit(128 + signal.SIGINT)
122 changes: 50 additions & 72 deletions Tests/iaas/mandatory-services/mandatory-iaas-services.py
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
#!/usr/bin/env python3
"""Mandatory APIs checker
This script retrieves the endpoint catalog from Keystone using the OpenStack
SDK and checks whether all mandatory APi endpoints, are present.
Expand Down Expand Up @@ -26,54 +27,30 @@
block_storage_service = ["volume", "volumev3", "block-storage"]


def connect(cloud_name: str) -> openstack.connection.Connection:
"""Create a connection to an OpenStack cloud
:param string cloud_name:
The name of the configuration to load from clouds.yaml.
:returns: openstack.connnection.Connection
"""
return openstack.connect(
cloud=cloud_name,
)


def check_presence_of_mandatory_services(cloud_name: str, s3_credentials=None):
try:
connection = connect(cloud_name)
services = connection.service_catalog
except Exception as e:
print(str(e))
raise Exception(
f"Connection to cloud '{cloud_name}' was not successfully. "
f"The Catalog endpoint could not be accessed. "
f"Please check your cloud connection and authorization."
)
def check_presence_of_mandatory_services(conn: openstack.connection.Connection, s3_credentials=None):
services = conn.service_catalog

if s3_credentials:
mandatory_services.remove("object-store")
for svc in services:
svc_type = svc['type']
if svc_type in mandatory_services:
mandatory_services.remove(svc_type)
continue
if svc_type in block_storage_service:
elif svc_type in block_storage_service:
block_storage_service.remove(svc_type)

bs_service_not_present = 0
if len(block_storage_service) == 3:
# neither block-storage nor volume nor volumev3 is present
# we must assume, that there is no volume service
logger.error("FAIL: No block-storage (volume) endpoint found.")
logger.error("No block-storage (volume) endpoint found.")
mandatory_services.append(block_storage_service[0])
bs_service_not_present = 1
if not mandatory_services:
# every mandatory service API had an endpoint
return 0 + bs_service_not_present
else:
# there were multiple mandatory APIs not found
logger.error(f"FAIL: The following endpoints are missing: "
f"{mandatory_services}")
return len(mandatory_services) + bs_service_not_present
if mandatory_services:
# some mandatory APIs were not found
logger.error(f"The following endpoints are missing: "
f"{', '.join(mandatory_services)}.")
return len(mandatory_services) + bs_service_not_present


def list_containers(conn):
Expand Down Expand Up @@ -167,8 +144,8 @@ def s3_from_ostack(creds, conn, endpoint):
# pass


def check_for_s3_and_swift(cloud_name: str, s3_credentials=None):
# If we get credentials we assume, that there is no Swift and only test s3
def check_for_s3_and_swift(conn: openstack.connection.Connection, s3_credentials=None):
# If we get credentials, we assume that there is no Swift and only test s3
if s3_credentials:
try:
s3 = s3_conn(s3_credentials)
Expand All @@ -183,58 +160,46 @@ def check_for_s3_and_swift(cloud_name: str, s3_credentials=None):
if s3_buckets == [TESTCONTNAME]:
del_bucket(s3, TESTCONTNAME)
# everything worked, and we don't need to test for Swift:
print("SUCCESS: S3 exists")
logger.info("SUCCESS: S3 exists")
return 0
# there were no credentials given, so we assume s3 is accessable via
# the service catalog and Swift might exist too
try:
connection = connect(cloud_name)
connection.authorize()
except Exception as e:
print(str(e))
raise Exception(
f"Connection to cloud '{cloud_name}' was not successfully. "
f"The Catalog endpoint could not be accessed. "
f"Please check your cloud connection and authorization."
)
s3_creds = {}
try:
endpoint = connection.object_store.get_endpoint()
except Exception as e:
logger.error(
f"FAIL: No object store endpoint found in cloud "
f"'{cloud_name}'. No testing for the s3 service possible. "
f"Details: %s", e
endpoint = conn.object_store.get_endpoint()
except Exception:
logger.exception(
"No object store endpoint found. No testing for the s3 service possible."
)
return 1
# Get S3 endpoint (swift) and ec2 creds from OpenStack (keystone)
s3_from_ostack(s3_creds, connection, endpoint)
s3_from_ostack(s3_creds, conn, endpoint)
# Overrides (var names are from libs3, in case you wonder)
s3_from_env(s3_creds, "HOST", "S3_HOSTNAME", "https://")
s3_from_env(s3_creds, "AK", "S3_ACCESS_KEY_ID")
s3_from_env(s3_creds, "SK", "S3_SECRET_ACCESS_KEY")

s3 = s3_conn(s3_creds, connection)
s3 = s3_conn(s3_creds, conn)
s3_buckets = list_s3_buckets(s3)
if not s3_buckets:
s3_buckets = create_bucket(s3, TESTCONTNAME)
assert s3_buckets

# If we got till here, s3 is working, now swift
swift_containers = list_containers(connection)
swift_containers = list_containers(conn)
# if not swift_containers:
# swift_containers = create_container(connection, TESTCONTNAME)
# swift_containers = create_container(conn, TESTCONTNAME)
result = 0
if Counter(s3_buckets) != Counter(swift_containers):
print("WARNING: S3 buckets and Swift Containers differ:\n"
f"S3: {sorted(s3_buckets)}\nSW: {sorted(swift_containers)}")
logger.warning("S3 buckets and Swift Containers differ:\n"
f"S3: {sorted(s3_buckets)}\nSW: {sorted(swift_containers)}")
result = 1
else:
print("SUCCESS: S3 and Swift exist and agree")
logger.info("SUCCESS: S3 and Swift exist and agree")
# Clean up
# FIXME: Cleanup created EC2 credential
# if swift_containers == [TESTCONTNAME]:
# del_container(connection, TESTCONTNAME)
# del_container(conn, TESTCONTNAME)
# Cleanup created S3 bucket
if s3_buckets == [TESTCONTNAME]:
del_bucket(s3, TESTCONTNAME)
Expand Down Expand Up @@ -266,34 +231,47 @@ def main():
help="Enable OpenStack SDK debug logging"
)
args = parser.parse_args()
logging.basicConfig(
format="%(levelname)s: %(message)s",
level=logging.DEBUG if args.debug else logging.INFO,
)
openstack.enable_logging(debug=args.debug)

# parse cloud name for lookup in clouds.yaml
cloud = os.environ.get("OS_CLOUD", None)
if args.os_cloud:
cloud = args.os_cloud
assert cloud, (
"You need to have the OS_CLOUD environment variable set to your cloud "
"name or pass it via --os-cloud"
)
cloud = args.os_cloud or os.environ.get("OS_CLOUD", None)
if not cloud:
raise RuntimeError(
"You need to have the OS_CLOUD environment variable set to your "
"cloud name or pass it via --os-cloud"
)

s3_credentials = None
if args.s3_endpoint:
if (not args.s3_access) or (not args.s3_access_secret):
print("WARNING: test for external s3 needs access key and access secret.")
logger.warning("test for external s3 needs access key and access secret.")
s3_credentials = {
"AK": args.s3_access,
"SK": args.s3_access_secret,
"HOST": args.s3_endpoint
}
elif args.s3_access or args.s3_access_secret:
print("WARNING: access to s3 was given, but no endpoint provided.")
logger.warning("access to s3 was given, but no endpoint provided.")

result = check_presence_of_mandatory_services(cloud, s3_credentials)
result = result + check_for_s3_and_swift(cloud, s3_credentials)
with openstack.connect(cloud) as conn:
result = check_presence_of_mandatory_services(conn, s3_credentials)
result += check_for_s3_and_swift(conn, s3_credentials)

print('service-apis-check: ' + ('PASS', 'FAIL')[min(1, result)])

return result


if __name__ == "__main__":
main()
try:
sys.exit(main())
except SystemExit:
raise
except BaseException as exc:
logging.debug("traceback", exc_info=True)
logging.critical(str(exc))
sys.exit(1)
Loading

0 comments on commit 48ba4d7

Please sign in to comment.