Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Code coverage metrics aren't uploaded when build fails due to low code coverage #2451

Closed
dbwiddis opened this issue May 16, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@dbwiddis
Copy link
Member

What is the bug?

I'm trying to improve code coverage on #2430. The build is passing all the unit and integ tests, but is failing on lack of coverage:

> Task :opensearch-ml-plugin:test
> Task :opensearch-ml-plugin:jacocoTestReport
[ant:jacocoReport] Rule violated for class org.opensearch.ml.sdkclient.RemoteClusterIndicesClient: branches covered ratio is 0.5, but expected minimum is 0.7
[ant:jacocoReport] Rule violated for class org.opensearch.ml.sdkclient.LocalClusterIndicesClient: branches covered ratio is 0.5, but expected minimum is 0.7

I'm covering about half of my branches, but honestly I don't know which ones I'm missing.

The code coverage report would tell me, but it doesn't run, because it's placed after the test in the CI yaml.

      - name: Build and Run Tests
        id: step-build-test-linux
        run: |
          // unit and integ tests

      - name: Upload Coverage Report
        uses: codecov/codecov-action@v1

This presents an extremely frustrating experience for a developer.

How can one reproduce the bug?
Steps to reproduce the behavior:

  1. Submit a PR with a file that does not meet minimum code coverage requirements but otherwise passes tests.
  2. Observe a lack of coverage reports.

What is the expected behavior?

When the specific reason for the build failure is lack of coverage, the specific lines/branches missed should be available to the developer somewhere.

What is your host/environment?

  • OS: Both Linux and WIndows CI

Do you have any additional context?

If codecov was placed in a separate workflow, it would always run. This gives the added benefit of ensuring the coverage comment is (one of) the first one on the PR and easier for reviewers to find. It would still fail that other workflow, although not quite as visibly as the tests.

Alternately if the desire is to keep the entire test failing on low coverage, the jacocoTestReport gradle task could be moved until after the coverage run in the workflow.

@dbwiddis dbwiddis added bug Something isn't working untriaged labels May 16, 2024
@dhrubo-os dhrubo-os self-assigned this May 21, 2024
@dhrubo-os dhrubo-os moved this to On-deck in ml-commons projects May 21, 2024
@dhrubo-os
Copy link
Collaborator

Cut an issue to the infra team for token: opensearch-project/.github#270

Raised the PR to downgrade codecov action: #3409

Now we can see the codecoveage report: Example PR

Closing the issue.

@github-project-automation github-project-automation bot moved this from On-deck to Done in ml-commons projects Jan 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Development

No branches or pull requests

2 participants