Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(.github/codecov): Ignore aws-lc-rs-fips for codecov #2023

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

dongcarl
Copy link

@dongcarl dongcarl commented Oct 29, 2024

An attempt to fix https://github.com/quinn-rs/quinn/actions/runs/11562272211/job/32183071070 on main and (unreleased) 0.5.6

@dongcarl
Copy link
Author

dongcarl commented Oct 29, 2024

It seems from the CI failure that code coverage builds for Windows for aws-lc-fips-sys is more complicated than expected.

More resources:

@mxinden @Ralith @djc I guess a decision has to be made if full coverage for aws-lc-fips-sys is worth the build system troubles

Copy link
Member

@djc djc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What goal are you trying to achieve here?

@dongcarl
Copy link
Author

@djc
Copy link
Member

djc commented Nov 5, 2024

@mxinden @Ralith @djc I guess a decision has to be made if full coverage for aws-lc-fips-sys is worth the build system troubles

I think we should definitely fix it so coverage builds can succeed (I'm surprised the main CI failures don't generate any notifications for me...). I think rustls has this working so it shouldn't be too hard to fix?

https://github.com/rustls/rustls/blob/main/.github/workflows/build.yml#L51

@dongcarl dongcarl force-pushed the 2024-10-fix-fips-golang branch from 6d859e2 to 80a485b Compare January 2, 2025 20:20
@dongcarl dongcarl changed the title fix(.github/codecov): Install golang for fips codecov fix(.github/codecov): Install deps for fips codecov Jan 2, 2025
@dongcarl
Copy link
Author

dongcarl commented Jan 2, 2025

@djc Should be good to go now. Checks are passing: https://github.com/quinn-rs/quinn/actions/runs/12588265064

I can drop the last commit (meant to be temporary just for testing) if this looks good.

@dongcarl dongcarl marked this pull request as ready for review January 2, 2025 22:35
Copy link
Collaborator

@gretchenfrage gretchenfrage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah just copying what Rustls is doing makes sense, and the second commit which makes the coverage CI run in PRs combined with the fact that CI passed for this PR seems to indicate that the change works. Thank you for working on this irksome thing!

(We must remember not to merge until the second commit is removed.)

.github/workflows/codecov.yml Outdated Show resolved Hide resolved
@djc
Copy link
Member

djc commented Jan 6, 2025

So looking at this more, maybe we should exclude fips from the features used for coverage? The feature has no effect on Quinn itself, so it should not change anything about coverage. The disadvantage would be that we sort of need to keep features in sync manually, but IMO that might be better than showing everything down for a relatively niche use case.

@dongcarl
Copy link
Author

dongcarl commented Jan 6, 2025

So looking at this more, maybe we should exclude fips from the features used for coverage? The feature has no effect on Quinn itself, so it should not change anything about coverage. The disadvantage would be that we sort of need to keep features in sync manually, but IMO that might be better than showing everything down for a relatively niche use case.

Yeah that was my initial thinking. Let me know what you guys decide and I can adjust this PR.

@Ralith
Copy link
Collaborator

Ralith commented Jan 6, 2025

+1 in favor of excluding it, since our CI is getting pretty big already.

@dongcarl dongcarl force-pushed the 2024-10-fix-fips-golang branch 2 times, most recently from ff405ef to c109d60 Compare January 8, 2025 22:10
@dongcarl dongcarl force-pushed the 2024-10-fix-fips-golang branch from c109d60 to 19273f5 Compare January 8, 2025 22:13
@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.89%. Comparing base (f5b1ec7) to head (19273f5).
Report is 3 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2023      +/-   ##
==========================================
+ Coverage   77.43%   77.89%   +0.46%     
==========================================
  Files          69       71       +2     
  Lines       18167    18722     +555     
==========================================
+ Hits        14067    14584     +517     
- Misses       4100     4138      +38     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dongcarl
Copy link
Author

dongcarl commented Jan 8, 2025

Updated to include all features except for aws-lc-rs-fips and rustls-aws-lc-rs-fips by parsing cargo metadata with jq. Let me know if this looks good.

Note that the last commit needs to be dropped.

@dongcarl dongcarl changed the title fix(.github/codecov): Install deps for fips codecov fix(.github/codecov): Ignore aws-lc-rs-fips for codecov Jan 8, 2025
@gretchenfrage gretchenfrage self-requested a review January 8, 2025 23:56
Copy link
Collaborator

@gretchenfrage gretchenfrage left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes sense, although I wish there was a way built in to cargo to do this. But it looks like there might not be.

Comment on lines +19 to +20
LLVMCOV_FEATURES=$(cargo metadata --format-version 1 --no-deps \
| jq -r '[ .packages[].features | keys[] | select(. != "default" and . != "aws-lc-rs-fips" and . != "rustls-aws-lc-rs-fips") ] | unique | join(",")')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just get the flattened version of this, without all the shell/jq magic? That stuff is unfamiliar (at least to me) and thus harder to maintain, and I don't think we change our features all that often.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants