Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/add cluster stacks plugin #800

Open
wants to merge 49 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
be0d581
Enable compliance tests to use plugins for cluster provisioning
tonifinger Sep 18, 2024
65f3041
Fix python style
tonifinger Sep 18, 2024
3b89b6a
Fix path to junit file
tonifinger Sep 20, 2024
f90ffdf
Inital integration of the plugin into the 'scs-test-runner.py'
tonifinger Oct 7, 2024
3b906c8
Adjust 'scs-test-runner.py' to handle kaas tests
tonifinger Oct 10, 2024
405deb1
Add ability to deploy clusters for different K8s versions
tonifinger Oct 14, 2024
b949bb1
Update `scs-test-runner.py`
tonifinger Oct 18, 2024
e93de5f
still draft: a few steps towards my vision
mbuechse Oct 23, 2024
120627e
Fixup!
tonifinger Oct 23, 2024
98c5efa
Fixup plugin kubeconfig file generation
tonifinger Oct 24, 2024
c6833df
Draft: split up clusterspec file
tonifinger Nov 4, 2024
b674aad
Merge to back to usage of one single clustersspec
tonifinger Nov 4, 2024
b977a0b
Rearange configuration files
tonifinger Nov 5, 2024
d166cdf
Fixup configuration files
tonifinger Nov 5, 2024
5ba05de
revert sonobuoy executor handling form this PR
tonifinger Nov 5, 2024
fb359e2
Apply cluster configuration file handling
tonifinger Nov 5, 2024
37b8018
Fixup
tonifinger Nov 5, 2024
254eaa5
Restructure abstract method handling in plugin
tonifinger Nov 6, 2024
c17537c
Fixup: change kube_plugin_config directory name
tonifinger Nov 6, 2024
a2e358c
Sort python imports
tonifinger Nov 7, 2024
b88749e
Fixup: use '__name__' for logging handler
tonifinger Nov 7, 2024
e9aa761
Update plugin kubeconfig handling
tonifinger Nov 7, 2024
1c427cc
Fixup: remove version parameter form delete
tonifinger Nov 11, 2024
928cd94
Initial implementation of the plugin for cluster-stacks
michal-gubricky Oct 11, 2024
e347075
Deploy cluster using cluster-stacks
michal-gubricky Oct 15, 2024
c9ee963
Fix python lint errors
michal-gubricky Oct 16, 2024
be8e7f3
Refactor the code
michal-gubricky Oct 21, 2024
2af6ee2
Replace sleep with function to wait for CAPI pods becomes ready
michal-gubricky Oct 22, 2024
2992fd6
Replace sleep with function to wait for CSO pods becomes ready
michal-gubricky Oct 24, 2024
8eb5687
Add ability to remove kubeconfig
michal-gubricky Oct 25, 2024
6a91fb3
Adapt cluster-stack plugin based on changes made in target branch and…
michal-gubricky Nov 2, 2024
bcf5f0f
Remove files that are not needed
michal-gubricky Nov 6, 2024
6959581
Update code and readme
michal-gubricky Nov 7, 2024
0174c5f
Fix error: no matching resources found when waiting for pods in workl…
michal-gubricky Nov 11, 2024
4b0e01b
Move example plugin configuration file under the playbooks/k8s_configs/
michal-gubricky Nov 13, 2024
a629cb9
Update Tests/kaas/plugin/plugin_cluster_stacks.py
michal-gubricky Nov 13, 2024
69a3d02
Merge two similiar functions into one wait_for_pods
michal-gubricky Nov 14, 2024
9fcbf2d
Update Tests/kaas/plugin/plugin_cluster_stacks.py
michal-gubricky Nov 14, 2024
e2ad244
Do not use os.environ.update to update the environment
michal-gubricky Nov 15, 2024
ab7489a
Rename/fix kubeconfig naming
michal-gubricky Nov 18, 2024
0f2f6c5
Add description to few functions and use only one function to wait fo…
michal-gubricky Nov 18, 2024
7b7740f
Update README and add option to change the namespace
michal-gubricky Nov 18, 2024
52516cb
Update Tests/kaas/plugin/plugin_cluster_stacks.py
michal-gubricky Nov 19, 2024
574e7c3
Handle the kubeconfig_mgmnt file in the plugin internally
michal-gubricky Nov 19, 2024
1c3a0d9
Remove mess after huge rebase
michal-gubricky Nov 20, 2024
8b893b8
Update Tests/kaas/plugin/plugin_cluster_stacks.py
michal-gubricky Nov 20, 2024
70f580c
Fix lint errors
michal-gubricky Nov 20, 2024
7925baa
Merge branch 'main' into feat/add-cluster-stacks-plugin
michal-gubricky Nov 22, 2024
c7837ad
Add wait for the clusterstack resource
michal-gubricky Nov 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ node_modules
Tests/kaas/results/
Tests/kaas/kaas-sonobuoy-tests/results/
*.tar.gz
.pytest-kind/
80 changes: 73 additions & 7 deletions Tests/kaas/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

## Development environment

### requirements
### Requirements

* [docker](https://docs.docker.com/engine/install/)
* [kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)
* [sonobuoy](https://sonobuoy.io/docs/v0.57.1/#installation)

### setup for development
### Setup for Development

1. Generate python 3.10 env
1. **Generate python 3.10 env**

```bash
sudo apt-get install python3.10-dev
Expand All @@ -22,17 +22,83 @@
(venv) python3.10 -m pip --version
```

2. Install dependencies:
2. **Install dependencies:**

```bash
(venv) pip install pip-tools
(venv) pip-compile requirements.in
(venv) pip-sync requirements.txt
```

3. Set environment variables and launch the process:
3. **Create a Cluster**

To create a cluster, run:

```bash
(venv) export CLUSTER_PROVIDER="kind"
(venv) python run.py
(venv) python run_plugin.py create <plugin-name> ../clusterspec.yaml
```

## Setting up and running the cluster-stacks plugin

This section covers how to configure and run the `cluster-stacks` plugin to provision Kubernetes clusters and perform conformance tests.

### Requirements for cluster-stacks plugin

In addition to the core requirements, ensure the following are set up:

* An **OpenStack** environment configured and accessible.
* A `clouds.yaml` file defining OpenStack credentials and endpoints.
* An installing the envsubst via this command:

```bash
GOBIN=/tmp go install github.com/drone/envsubst/v2/cmd/envsubst@latest
```

### Environment Variables

Before running the `cluster-stacks` plugin, you need to set up the following environment variable:

* **GIT_ACCESS_TOKEN**: This token is required for Git operations, especially if your repository is private.

To set the `GIT_ACCESS_TOKEN`, run the following command in your terminal:

```bash
export GIT_ACCESS_TOKEN=<your-github-token>
```

### Configuring config file for cluster-stacks plugin

The config file is used to set parameters for creating a Kubernetes cluster with the `cluster-stacks` plugin. This file allows you to specify details related to the cluster-stack, Git integration, and cluster configurations. These optional parameters are grouped below by their category.

#### Cluster-Stack Related Parameters

These parameters configure specific settings for the cluster-stack:

```yaml
cs_name: <cs_name> # Cluster Stack Name, default: "scs"
cs_version: <cs_version> # Cluster Stack Version
cs_channel: <cs_channel> # Release channel
cs_cloudname: <cs_cloudname> # Cloud name from OpenStack clouds.yaml
cs_namespace: "default" # Namespace for the Cluster Stack deployment
```

#### Git-Related Parameters

The [Cluster Stack Operator](https://github.com/sovereignCloudStack/cluster-stack-operator/) (CSO) utilizing the [Cluster Stack Provider OpenStack](https://github.com/SovereignCloudStack/cluster-stacks/tree/main/providers/openstack) (CSPO) must be directed to the Cluster Stacks repository housing releases for the OpenStack provider. Modify the following parameters if you wish to redirect CSO and CSPO to an alternative Git repository

```yaml
git_provider: <git_provider> # Git provider, default: "github"
git_org_name: <git_org_name> # Organization name on Git provider, default: "SovereignCloudStack"
git_repo_name: <git_repo_name> # Repository name on Git provider, default: "cluster-stacks"
```

#### Cluster Parameters

Set these parameters to customize the configuration for your cluster.

```yaml
cs_pod_cidr: <cs_pod_cidr> # Pod CIDR for networking
cs_service_cidr: <cs_service_cidr> # Service CIDR for networking
cs_external_id: <cs_external_id> # External ID for the Cluster Stack
cs_k8s_patch_version: <cs_k8s_patch_version> # Kubernetes patch version to use
```
35 changes: 35 additions & 0 deletions Tests/kaas/plugin/cluster.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Cluster resource template
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: ${CS_CLUSTER_NAME}
namespace: ${CS_NAMESPACE}
labels:
managed-secret: cloud-config
spec:
clusterNetwork:
pods:
cidrBlocks:
- ${CS_POD_CIDR}
serviceDomain: cluster.local
services:
cidrBlocks:
- ${CS_SERVICE_CIDR}
topology:
variables:
- name: controller_flavor
value: "SCS-2V-4-50"
- name: worker_flavor
value: "SCS-2V-4-50"
- name: external_id
value: ${CS_EXTERNAL_ID}
class: ${CS_CLASS_NAME}
controlPlane:
replicas: 1
version: v${CS_K8S_VERSION}.${CS_K8S_PATCH_VERSION}
workers:
machineDeployments:
- class: default-worker
failureDomain: nova
name: default-worker
replicas: 3
30 changes: 30 additions & 0 deletions Tests/kaas/plugin/clusterstack.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Cluster-stack and OpenStack cluster-stack release resource templates
apiVersion: clusterstack.x-k8s.io/v1alpha1
kind: ClusterStack
metadata:
name: clusterstack
namespace: ${CS_NAMESPACE}
spec:
provider: openstack
name: ${CS_NAME}
kubernetesVersion: "${CS_K8S_VERSION}"
channel: ${CS_CHANNEL}
autoSubscribe: false
providerRef:
apiVersion: infrastructure.clusterstack.x-k8s.io/v1alpha1
kind: OpenStackClusterStackReleaseTemplate
name: cspotemplate
versions:
- ${CS_VERSION}
---
apiVersion: infrastructure.clusterstack.x-k8s.io/v1alpha1
kind: OpenStackClusterStackReleaseTemplate
metadata:
name: cspotemplate
namespace: ${CS_NAMESPACE}
spec:
template:
spec:
identityRef:
kind: Secret
name: ${CS_SECRETNAME}
3 changes: 2 additions & 1 deletion Tests/kaas/plugin/interface.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,10 @@ def create_cluster(self, cluster_name, version, kubeconfig_filepath):
"""
raise NotImplementedError

def delete_cluster(self, cluster_name):
def delete_cluster(self, cluster_name, kubeconfig_filepath):
"""
This method is to be called in order to unprovision a cluster
:param: cluster_name:
:param: kubeconfig_filepath:
"""
raise NotImplementedError
19 changes: 19 additions & 0 deletions Tests/kaas/plugin/plugin-cluster-stacks-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# This is an example of the config for a cluster-stacks plugins

clouds_yaml_path: "~/.config/openstack/clouds.yaml" # Path to OpenStack clouds.yaml file
cs_name: "scs" # Cluster Stack Name
cs_version: "v1" # Cluster Stack Version
cs_channel: "stable" # Release channel
cs_cloudname: "openstack" # Cloud name from OpenStack clouds.yaml
cs_namespace: "default" # Namespace for the Cluster Stack deployment

# Git Configuration
git_provider: "github" # Git provider (e.g., GitHub, GitLab)
git_org_name: "SovereignCloudStack" # Organization name on Git provider
git_repo_name: "cluster-stacks" # Repository name on Git provider

# Cluster Information
cs_pod_cidr: "192.168.0.0/16" # Pod CIDR for networking
cs_service_cidr: "10.96.0.0/12" # Service CIDR for networking
cs_external_id: "ebfe5546-f09f-4f42-ab54-094e457d42ec" # External ID for the Cluster Stack
cs_k8s_patch_version: "6" # Kubernetes patch version to use
Loading
Loading