Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve server logic #235

Merged
merged 10 commits into from
Nov 21, 2024
Merged

Improve server logic #235

merged 10 commits into from
Nov 21, 2024

Conversation

nadzyah
Copy link
Collaborator

@nadzyah nadzyah commented Nov 6, 2024

This PR updates the server logic after testing it on certified machines using scripts in #234. The patches contain the following fixes:

  • Use a list of BIOSes instead of first(). Since we match only bios vendor and version, there might be multiple BIOSes with different revisions but the same version. One of them may belong to the matching machine. So, now we use all matching BIOSes instead of getting the first one. Additionally, we don't match other fields like revision since firmware may receive revision updates pretty frequently, but it still allows us to consider hardware as potentially certified since we're still working with the same BIOS version
  • Checking the board version turned out to be a strict validation. After the discussion with the team, it was decided to check only the board vendor and model
  • It turned out that the lsb_release command used by the client to collect the OS version returns the string in format 22.04 without the LTS suffix. However, C3 stores it with the LTS suffix. The logic was added to remove this suffix while importing the data from C3
  • The hardware mismatches are logged now

Also, the tests were revised/added to cover the implemented logic

Copy link

github-actions bot commented Nov 11, 2024

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
654 632 97% 70% 🟢

New Files

No new covered files...

Modified Files

File Coverage Status
server/hwapi/data_models/repository.py 99% 🟢
server/hwapi/endpoints/certification/certification.py 100% 🟢
server/hwapi/endpoints/certification/logic.py 87% 🟢
server/hwapi/endpoints/certification/response_validators.py 100% 🟢
server/hwapi/external/c3/client.py 90% 🟢
TOTAL 95% 🟢

updated for commit: 3b7e775 by action🐍

@nadzyah nadzyah marked this pull request as ready for review November 11, 2024 15:59
@nadzyah nadzyah requested review from omar-selo and plars November 11, 2024 15:59
@nadzyah nadzyah force-pushed the improve-server-logic branch from e6717f2 to 89e4b11 Compare November 15, 2024 16:52
@omar-selo
Copy link

Reading the PR description, I wonder if it's even necessary to match BIOS when checking if a hardware is certified. As in does the BIOS used affect whether Ubuntu would run fine or not?

Copy link

@omar-selo omar-selo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes. Just had a few comments. Main ones are around adding a testing helper class and avoiding logging for testing

Comment on lines +74 to +83
logging.error(
(
"Hardware cannot be found. Machine vendor: %s, model: %s"
", board model: %s, board version: %s, bios version: %s"
),
system_info.vendor,
system_info.model,
system_info.board.product_name,
system_info.board.version,
system_info.bios.version if system_info.bios else None,
)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This endpoint checks whether a hardware is certified or not. If it's not certified then it responds with NotCertifiedResponse. If so then it is expected behavior to find not certified device where the board doesn't match for instance. Hence it's not an error. So perhaps it's not correct to log an error here. Additionally, this is the API any logs made here are not shown to the client so we shouldn't be logging anything apart from real errors, warnings or perhaps debug info.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matias specifically asked for logging hardware mismatch data since we'll probably have to adjust the logic according to, for instance, how many certified machines run a newer bios version, different from which we used for issuing a certificate

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds like we need to store this data to be able to query it rather than just log it. I'm not sure if simply logging will help us get such information. @mz2 wdyt?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, it probably is better that way. I was imagining deriving metrics from logs, but explicitly storing the full data is probably a better idea.

Copy link
Collaborator Author

@nadzyah nadzyah Nov 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After discussing it with @omar-selo, we've decided to add this change in another PR and create a table for storing hardware mismatches
Here is the Jira ticket for it: https://warthogs.atlassian.net/browse/C3-945

server/tests/endpoints/certification/test_certification.py Outdated Show resolved Hide resolved
server/tests/endpoints/certification/test_certification.py Outdated Show resolved Hide resolved
Comment on lines 174 to 175
assert "Hardware cannot be found" in caplog.text
assert "bios version: 1.0.15" in caplog.text

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that the logs are being used to test that the reason hardware is "Not Seen" is what's expected of this test. It's great that we're testing this. But I suggest that instead of depending on logs, we test it directly. This can be done by refactoring the logic that checks if hardware is certified in it's own function. This function would be used by the controller. It can return not only the status but also the reason. Then we can write tests for that function to make sure the reason matches what we expected.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matias asked to check that if there is a hardware mismatch, we log this information

server/tests/endpoints/certification/test_certification.py Outdated Show resolved Hide resolved
server/hwapi/endpoints/certification/logic.py Outdated Show resolved Hide resolved
server/hwapi/endpoints/certification/logic.py Outdated Show resolved Hide resolved
@nadzyah nadzyah force-pushed the improve-server-logic branch from f0bd6c5 to 4969058 Compare November 20, 2024 10:12
@nadzyah nadzyah requested a review from omar-selo November 20, 2024 12:22
@nadzyah nadzyah force-pushed the improve-server-logic branch from ce66ff9 to 57ce302 Compare November 20, 2024 15:03
@nadzyah nadzyah force-pushed the improve-server-logic branch from 57ce302 to 70a7a44 Compare November 21, 2024 11:07
Copy link

@omar-selo omar-selo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! and thanks addressing my comments!

@nadzyah nadzyah merged commit 16fa1c6 into main Nov 21, 2024
4 checks passed
@nadzyah nadzyah deleted the improve-server-logic branch November 21, 2024 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants