Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(csaf): finalize db schema #1076

Merged
merged 14 commits into from
Mar 25, 2024
Merged

fix(csaf): finalize db schema #1076

merged 14 commits into from
Mar 25, 2024

Conversation

psegedy
Copy link
Member

@psegedy psegedy commented Feb 19, 2024

RHINENG-7862

further changes will be added once #1073 is completed

Complete Schema

csaf_schema_additions
(it's not cve_detail table but cve, I'll fix the diagram later, code should be correct)

Secure Coding Practices Checklist GitHub Link

Secure Coding Checklist

  • Input Validation
  • Output Encoding
  • Authentication and Password Management
  • Session Management
  • Access Control
  • Cryptographic Practices
  • Error Handling and Logging
  • Data Protection
  • Communication Security
  • System Configuration
  • Database Security
  • File Management
  • Memory Management
  • General Coding Practices

Copy link

jira-linking bot commented Feb 19, 2024

@psegedy psegedy force-pushed the schema branch 7 times, most recently from 12d7bda to 371af61 Compare February 19, 2024 21:54
@codecov-commenter
Copy link

codecov-commenter commented Feb 19, 2024

Codecov Report

Attention: Patch coverage is 82.18391% with 62 lines in your changes are missing coverage. Please review.

Project coverage is 62.80%. Comparing base (45fdf5e) to head (1f1a428).

Files Patch % Lines
vmaas/reposcan/database/csaf_store.py 82.74% 34 Missing ⚠️
vmaas/reposcan/redhatcsaf/csaf_controller.py 59.25% 11 Missing ⚠️
vmaas/common/batch_list.py 0.00% 7 Missing ⚠️
vmaas/common/config.py 0.00% 4 Missing ⚠️
vmaas/reposcan/redhatcsaf/modeling.py 96.93% 3 Missing ⚠️
vmaas/common/date_utils.py 0.00% 1 Missing ⚠️
vmaas/common/logging_utils.py 0.00% 1 Missing ⚠️
vmaas/common/rpm_utils.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1076      +/-   ##
==========================================
+ Coverage   61.66%   62.80%   +1.14%     
==========================================
  Files          69       69              
  Lines        6445     6652     +207     
==========================================
+ Hits         3974     4178     +204     
- Misses       2471     2474       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@psegedy psegedy force-pushed the schema branch 10 times, most recently from ac33a09 to a7830af Compare March 11, 2024 20:30
@psegedy psegedy requested a review from jdobes March 13, 2024 16:27
@psegedy psegedy force-pushed the schema branch 3 times, most recently from 3e2df7b to 81617fd Compare March 14, 2024 10:11
@psegedy psegedy force-pushed the schema branch 3 times, most recently from 70e20e8 to e9643c3 Compare March 15, 2024 14:11
@psegedy psegedy force-pushed the schema branch 4 times, most recently from 96cbfca to 533b134 Compare March 18, 2024 15:04
@psegedy psegedy requested a review from jdobes March 19, 2024 11:25
@psegedy psegedy marked this pull request as draft March 19, 2024 12:34
@psegedy psegedy force-pushed the schema branch 3 times, most recently from 3124ea5 to 38aea44 Compare March 19, 2024 13:05
@psegedy psegedy marked this pull request as ready for review March 19, 2024 13:34
@psegedy
Copy link
Member Author

psegedy commented Mar 19, 2024

We will need to update cvemap for ephemeral because cves from csaf are missing in the current cvemap used for testing

@mtclinton
Copy link
Contributor

/retest

@psegedy
Copy link
Member Author

psegedy commented Mar 21, 2024

Based on logs we still have an issue - it tried to insert duplicate row, I will investigate

2024-03-20 11:39:50,952:ERROR:vmaas.reposcan.database.object_store:Failure while inserting data into csaf_product: 'Traceback (most recent call last):\n  File "/vmaas/vmaas/reposcan/database/csaf_store.py", line 262, in _insert_missing_products\n    rows = execute_values(\n           ^^^^^^^^^^^^^^^\n  File "/usr/local/lib64/python3.11/site-packages/psycopg2/extras.py", line 1299, in execute_values\n    cur.execute(b\'\'.join(parts))\npsycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "csaf_product_cpe_id_package_name_id_idx"\nDETAIL:  Key (cpe_id, package_name_id)=(907, 12550) already exists.\n'|

Edit:
It happens when there is the same cpe mapped to 2 different product_ids (used only to map vulnerability from vulnerabilities.product_status.known_affected to cpe)
image

@psegedy psegedy force-pushed the schema branch 3 times, most recently from 7f739e2 to 811e10d Compare March 21, 2024 18:06
@jdobes
Copy link
Member

jdobes commented Mar 25, 2024

/retest

@jdobes jdobes merged commit 31e1e4a into RedHatInsights:master Mar 25, 2024
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants