e2e: introduce upgrade suite

[rbaturov]

Starting in version 4.18, NROP MachineConfigs containing the custom SELinux policy are expected to be removed unless a specific annotation is set in the NUMAResourcesOperator CR to enforce the use of the custom (legacy) SELinux policy. To ensure this behavior, we added a test that verifies MachineConfigs are removed when the annotation is absent in the CR.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[rbaturov]

/test all

[rbaturov]

/retest-required

[rbaturov]

@ffromani @Tal-or @shajmakh
Ready for another review iteration

[ffromani]

what I'm missing is how this suite is meant to be used: preconditions, expectations...

scratch that, just need to re-read the description better

[rbaturov]

/retest-required

[ffromani]

/retest-required

won't pass till #1099 merges, or till the real root issue gets fixed (which we have no visibility about)

[ffromani]

once #1111 lands, we can reimplement the version check using the info bundled in the operator container image, avoiding any extra deps.
To find the operator pod we can add something

$ cat test/utils/deploy/find.go 
/*
 * Copyright 2024 Red Hat, Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package deploy

import (
	"context"
        "errors"
	"fmt"

	corev1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/klog/v2"

	"sigs.k8s.io/controller-runtime/pkg/client"

	nropv1 "github.com/openshift-kni/numaresources-operator/api/numaresourcesoperator/v1"
)

func FindNUMAResourcesOperatorPod(ctx context.Context, cli client.Client, nrop *nropv1.NUMAResourcesOperator) (*corev1.Pod, error) {
	if len(nrop.Status.NodeGroups) < 1 {
		return nil, errors.New("node groups not reported, nothing to do")
	}
	// nrop places all daemonsets in the same namespace on which it resides, so any group is fine
	namespace := nrop.Status.NodeGroups[0].DaemonSet.Namespace // shortcut
	klog.InfoS("NROP pod", "namespace", namespace)

	sel, err := metav1.LabelSelectorAsSelector(&metav1.LabelSelector{
		MatchLabels: map[string]string{
			"app": "numaresources-operator",
		},
	})
	if err != nil {
		return nil, err
	}
	klog.InfoS("NROP pod", "selector", sel.String())

	podList := corev1.PodList{}
	err = cli.List(ctx, &podList, &client.ListOptions{Namespace: namespace, LabelSelector: sel})
	if err != nil {
		return nil, err
	}

	if len(podList.Items) != 1 {
		return nil, fmt.Errorf("unexpected number of pods found: %d", len(podList.Items))
	}

	return &podList.Items[0], nil
}

[rbaturov]

once #1111 lands, we can reimplement the version check using the info bundled in the operator container image, avoiding any extra deps. To find the operator pod we can add something

$ cat test/utils/deploy/find.go 
/*
 * Copyright 2024 Red Hat, Inc.
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package deploy

import (
	"context"
        "errors"
	"fmt"

	corev1 "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/klog/v2"

	"sigs.k8s.io/controller-runtime/pkg/client"

	nropv1 "github.com/openshift-kni/numaresources-operator/api/numaresourcesoperator/v1"
)

func FindNUMAResourcesOperatorPod(ctx context.Context, cli client.Client, nrop *nropv1.NUMAResourcesOperator) (*corev1.Pod, error) {
	if len(nrop.Status.NodeGroups) < 1 {
		return nil, errors.New("node groups not reported, nothing to do")
	}
	// nrop places all daemonsets in the same namespace on which it resides, so any group is fine
	namespace := nrop.Status.NodeGroups[0].DaemonSet.Namespace // shortcut
	klog.InfoS("NROP pod", "namespace", namespace)

	sel, err := metav1.LabelSelectorAsSelector(&metav1.LabelSelector{
		MatchLabels: map[string]string{
			"app": "numaresources-operator",
		},
	})
	if err != nil {
		return nil, err
	}
	klog.InfoS("NROP pod", "selector", sel.String())

	podList := corev1.PodList{}
	err = cli.List(ctx, &podList, &client.ListOptions{Namespace: namespace, LabelSelector: sel})
	if err != nil {
		return nil, err
	}

	if len(podList.Items) != 1 {
		return nil, fmt.Errorf("unexpected number of pods found: %d", len(podList.Items))
	}

	return &podList.Items[0], nil
}

@ffromani I'm not sure how I should use the buildInfo utility with this

[ffromani]

the idea would be to use a function like this to find the operator pod (from the test) and once in the operator pod trivially query (aka cat) buildinfo.json to learn how the operator was built without needing external libs, OLM access etc. etc.

[rbaturov]

the idea would be to use a function like this to find the operator pod (from the test) and once in the operator pod trivially query (aka cat) buildinfo.json to learn how the operator was built without needing external libs, OLM access etc. etc.

Thanks, much more clear now

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ffromani, rbaturov, Tal-or

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Tal-or,ffromani]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rbaturov]

/cherry-pick release-4.18

[openshift-cherrypick-robot]

@rbaturov: cannot checkout 4.18: error checking out "4.18": exit status 1 error: pathspec '4.18' did not match any file(s) known to git

In response to this:

/cherry-pick 4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-cherrypick-robot]

@rbaturov: new pull request created: #1129

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.