Green Builds Showing Despite Test Failures

[aparnamichael]

Even when tests are failing, the builds are showing as green.

PR- https://github.com/OpenLiberty/liberty-tools-vscode/actions/runs/12921114303?pr=478

[gilbysunil14]

The RunTests step in GHA is run through the script exec.sh. In mac machine, the tests are run through two commands test-mac-maven and test-mac-gradle in exec.sh file. See the below image:

The rc command returns a specific value indicating whether the previous operation was successful or not. Since the tests in mac machine are run through two commands stated above, and even if the first command is unsuccessful and the second command is successful, then the GHA will show a green build.

Proposed solution would be to run the tests in mac machine thorough a single command npm run test

[TrevCraw]

In the future, we may want to be able to run other kinds of tests. Typically, it is better to move towards being more modular and not less so. Is there not a way to capture the return code of individual npm commands? And then store those results in an array and check them at the end?

[gilbysunil14]

@TrevCraw as per your suggestion moving forward with a modular approach. Instead of having an array we will have a boolean to see if any of the commands that executed got failed and if yes the build will be exited with an error.