RP E2E Test Execution #3856

shubhadapaithankar · 2024-09-20T22:43:31Z

Which issue this PR addresses:

This PR addresses inefficiencies in our CI and E2E testing process. The current setup is slow and lacks full containerization and Hive integration for cluster management. By using Docker Compose and integrating Hive, this PR reduces runtime, simplifies orchestration, and improves testing reliability.

Fixes: ARO-9501

What this PR does / why we need it:

This PR introduces an E2E testing stage using Docker Compose to orchestrate the RP, Selenium, and E2E test services. The pipeline pulls the required RP image from Azure Container Registry (ACR) and ensures the correct environment is set up for the E2E tests. It helps validate the end-to-end functionality of the system and ensures that tests are running using containerized RP images.

Why we need it: The E2E stage ensures that the entire system is tested in a environment, using containerized components (including RP) and validating workflows through Selenium for browser-based testing.
Key benefits: This change helps automate the test execution process for the complete system, ensuring that every aspect of the application functions correctly in a containerized environment before deployment.

Key Points:

Improved Efficiency: This new E2E pipeline reduces the runtime for both CI and E2E stages by more than half compared to the current process.
Hive Integration: We’re now using Hive in the pipeline to create and manage clusters. Hive allows us to propagate required credentials to the installer, as outlined in Hive’s AWS provisioning guide (we apply similar functionality for Azure). This is a major step forward for MIWI, as it enhances our cluster setup, ensuring smoother and more reliable testing.
Deployment of Hive Operator: Deploying the Hive Operator in this process is another major step forward for MIWI, ensuring better control and integration of clusters.
Fully Dockerized Process: All services are run via Docker Compose, allowing us to orchestrate the setup with a single command, making the process simpler and more streamlined.
Image Management: Images for components like rp and e2e are created during the CI containerized stage and then pulled directly for use in the E2E stage.

Test plan for issue:

E2E tests are executed within the pipeline's new E2E stage using Docker Compose.
The tests are triggered automatically after the Containerized stage, ensuring seamless orchestration.

Is there any documentation that needs to be updated for this PR?

N/A

How do you know this will function as expected in production?

N/A

… responses and improve failure detection.

…startup behavior. Modified vpn logic again Added KUBECONFIG to test panic error again Removed the Kubeconfig veriable

…nsistent behavior. - Improved maintainability by defining versions as variables.

shubhadapaithankar · 2024-11-20T19:18:29Z

/azp run ci, e2e

azure-pipelines · 2024-11-20T19:18:44Z

Azure Pipelines successfully started running 2 pipeline(s).

jaitaiwan

LGTM, I had a small request to add a todo referencing the relevant JIRA where the skip is. Not necessary but I think a good addition. PR looks g.

docker-compose.yml

…shed updated configuration.

jaitaiwan

LGTM

shubhadapaithankar · 2024-11-22T01:05:02Z

/azp run ci, e2e

azure-pipelines · 2024-11-22T01:05:16Z

Azure Pipelines successfully started running 2 pipeline(s).

edisonLcardenas · 2024-11-22T04:40:29Z

hack/hive/hive-dev-install.sh

 main() {
 	log "enter hive installation"
 	local skip_deployments=${1:-"none"}

-	if [ ! -f "./hack/hive/hive-config/hive-deployment.yaml" ] || [ ! -d "./hack/hive/hive-config/crds" ] ; then
+	if [[ ! -f "./hack/hive/hive-config/hive-deployment.yaml" ]] || [[ ! -d "./hack/hive/hive-config/crds" ]]; then


What improvement does double bracket bring?

During pipeline testing, I found that the conditional on line 37 was failing when checking if PULL_SECRET is empty. Replacing [ ... ] with [[ ... ]] fixed the issue because [[ ... ]] handles empty values and special characters more reliably.

source : https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_07_01.html
example from our code base: https://github.com/openshift-metal3/dev-scripts/blob/ab8100cfd50f03633d71a70a8a1207622e9e9122/common.sh#L122

shubhadapaithankar marked this pull request as draft September 20, 2024 22:43

shubhadapaithankar added work-in-progress loki labels Sep 20, 2024

shubhadapaithankar force-pushed the shubhadapaithankar/ARO-9501-RP-E2E-Test branch 7 times, most recently from ed42f14 to f62f93f Compare September 23, 2024 18:39

shubhadapaithankar added 8 commits November 20, 2024 07:59

Improve reliability of E2E test pass/fail reporting in pipeline

c3aa772

Rmoved KUBECONFIG=aks.kubeconfig as its no longer needed

8a8562e

feat(pipeline): Decouple Hive operator deployment from CI/E2E pipelines

f7b0f37

Enhance healthchecks for vpn and portal services to validate HTTP 200…

9fba902

… responses and improve failure detection.

Refine for portal service to clarify rp dependency and ensure proper …

786576a

…startup behavior. Modified vpn logic again Added KUBECONFIG to test panic error again Removed the Kubeconfig veriable

Modified to pin versions of Docker-related packages (, , , , ) for co…

1cd2a31

…nsistent behavior. - Improved maintainability by defining versions as variables.

Pin Docker dependency versions for stable pipeline execution.

8967018

Removed unused code

877e361

shubhadapaithankar force-pushed the shubhadapaithankar/ARO-9501-RP-E2E-Test branch from 6062210 to 877e361 Compare November 20, 2024 16:00

slawande2 previously approved these changes Nov 20, 2024

View reviewed changes

jaitaiwan previously approved these changes Nov 22, 2024

View reviewed changes

docker-compose.yml Outdated Show resolved Hide resolved

docker-compose.yml Outdated Show resolved Hide resolved

Reverted interval to default, increased retries for stability, and pu…

ff4d82e

…shed updated configuration.

shubhadapaithankar dismissed stale reviews from jaitaiwan, slawande2, fahlmant, and kevinobriendotca via ff4d82e November 22, 2024 00:50

shubhadapaithankar requested review from kevinobriendotca, fahlmant, jaitaiwan and slawande2 November 22, 2024 00:59

jaitaiwan approved these changes Nov 22, 2024

View reviewed changes

edisonLcardenas reviewed Nov 22, 2024

View reviewed changes

slawande2 approved these changes Nov 22, 2024

View reviewed changes

fahlmant approved these changes Nov 22, 2024

View reviewed changes

fahlmant merged commit ed9f9c9 into master Nov 22, 2024
22 checks passed

shubhadapaithankar deleted the shubhadapaithankar/ARO-9501-RP-E2E-Test branch November 26, 2024 01:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RP E2E Test Execution #3856

RP E2E Test Execution #3856

shubhadapaithankar commented Sep 20, 2024 •

edited

Loading

shubhadapaithankar commented Nov 20, 2024

azure-pipelines bot commented Nov 20, 2024

jaitaiwan left a comment

jaitaiwan left a comment

shubhadapaithankar commented Nov 22, 2024

azure-pipelines bot commented Nov 22, 2024

edisonLcardenas Nov 22, 2024

shubhadapaithankar Nov 22, 2024 •

edited

Loading

RP E2E Test Execution #3856

RP E2E Test Execution #3856

Conversation

shubhadapaithankar commented Sep 20, 2024 • edited Loading

Which issue this PR addresses:

What this PR does / why we need it:

Key Points:

Test plan for issue:

Is there any documentation that needs to be updated for this PR?

How do you know this will function as expected in production?

shubhadapaithankar commented Nov 20, 2024

azure-pipelines bot commented Nov 20, 2024

jaitaiwan left a comment

Choose a reason for hiding this comment

jaitaiwan left a comment

Choose a reason for hiding this comment

shubhadapaithankar commented Nov 22, 2024

azure-pipelines bot commented Nov 22, 2024

edisonLcardenas Nov 22, 2024

Choose a reason for hiding this comment

shubhadapaithankar Nov 22, 2024 • edited Loading

Choose a reason for hiding this comment

shubhadapaithankar commented Sep 20, 2024 •

edited

Loading

shubhadapaithankar Nov 22, 2024 •

edited

Loading