Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CI failures related to security plugin download #2171

Merged
merged 20 commits into from
Jan 31, 2025

Conversation

cwperks
Copy link
Member

@cwperks cwperks commented Jan 29, 2025

Description

There were 2 issues that needed fixing:

  1. The download of the security plugin was failing due to using a deprecated -Ddest setting in the mvn command
  2. SAML Tests were failing because the backend couldn't fetch metadata. I had to switch to IPv6 address for the metadata url, but then ran into another issue where cypress couldn't understand IPv6. To fix that I wrote an interceptor to convert between IPv6 and localhost

Category

Maintenance

Issues Resolved

Resolves: #2168

Check List

  • New functionality includes testing
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Copy link

codecov bot commented Jan 29, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.46%. Comparing base (bc2c444) to head (2a986b2).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2171   +/-   ##
=======================================
  Coverage   71.46%   71.46%           
=======================================
  Files          97       97           
  Lines        2649     2649           
  Branches      403      403           
=======================================
  Hits         1893     1893           
  Misses        641      641           
  Partials      115      115           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@cwperks cwperks changed the title Attempt to fix CI failures related to security plugin download Fix CI failures related to security plugin download Jan 29, 2025
@cwperks cwperks added the backport 2.x backport to 2.x branch label Jan 29, 2025
RyanL1997
RyanL1997 previously approved these changes Jan 29, 2025
@cwperks
Copy link
Member Author

cwperks commented Jan 29, 2025

SAML failures may be related to opensearch-project/security#5038

@cwperks
Copy link
Member Author

cwperks commented Jan 29, 2025

SAML failures may be related to opensearch-project/security#5038

They are not related to this change.

Looks like the SAML tests have been silently failing with the same error that OpenSearch fails to read the node IdP's metadata.

Ref: https://productionresultssa11.blob.core.windows.net/actions-results/970b70a7-00bf-4f10-b4dc-dab541a49a58/workflow-job-run-0938180d-7dc4-505a-419f-7ddfa86bdfbf/logs/job/job-logs.txt?rsct=text%2Fplain&se=2025-01-29T22%3A43%3A58Z&sig=NW7EmwuDfNrgH0z9jbJVH9wWlUT%2FkyCw3R9sSsxiwYI%3D&ske=2025-01-30T06%3A51%3A33Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2025-01-29T18%3A51%3A33Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2025-01-05&sp=r&spr=https&sr=b&st=2025-01-29T22%3A33%3A53Z&sv=2025-01-05

2025-01-09T15:37:45.8088917Z [2025-01-09T15:37:22,125][ERROR][o.o.s.m.r.i.HTTPMetadataResolver] [smoketestnode] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Error retrieving metadata from http://localhost:7000/metadata: Connect to localhost:7000 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused
2025-01-09T15:37:45.8091007Z [2025-01-09T15:37:22,125][ERROR][o.o.s.m.r.i.AbstractReloadingMetadataResolver] [smoketestnode] Metadata Resolver SamlHTTPMetadataResolver com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator_1: Error occurred while attempting to refresh metadata from 'http://localhost:7000/metadata'
2025-01-09T15:37:45.8092347Z net.shibboleth.utilities.java.support.resolver.ResolverException: Error retrieving metadata from http://localhost:7000/metadata
2025-01-09T15:37:45.8093306Z 	at org.opensaml.saml.metadata.resolver.impl.HTTPMetadataResolver.fetchMetadata(HTTPMetadataResolver.java:239) ~[opensaml-saml-impl-4.3.2.jar:?]
2025-01-09T15:37:45.8094563Z 	at com.amazon.dlic.auth.http.saml.SamlHTTPMetadataResolver.access$001(SamlHTTPMetadataResolver.java:31) ~[opensearch-security-3.0.0.0-SNAPSHOT.jar:3.0.0.0-SNAPSHOT]
2025-01-09T15:37:45.8095709Z 	at com.amazon.dlic.auth.http.saml.SamlHTTPMetadataResolver.lambda$fetchMetadata$0(SamlHTTPMetadataResolver.java:44) ~[opensearch-security-3.0.0.0-SNAPSHOT.jar:3.0.0.0-SNAPSHOT]
2025-01-09T15:37:45.8096608Z 	at java.base/java.security.AccessController.doPrivileged(AccessController.java:571) ~[?:?]
2025-01-09T15:37:45.8097482Z 	at com.amazon.dlic.auth.http.saml.SamlHTTPMetadataResolver.fetchMetadata(SamlHTTPMetadataResolver.java:44) ~[opensearch-security-3.0.0.0-SNAPSHOT.jar:3.0.0.0-SNAPSHOT]
2025-01-09T15:37:45.8098771Z 	at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.refresh(AbstractReloadingMetadataResolver.java:364) [opensaml-saml-impl-4.3.2.jar:?]
2025-01-09T15:37:45.8100038Z 	at org.opensaml.saml.metadata.resolver.impl.AbstractReloadingMetadataResolver.initMetadataResolver(AbstractReloadingMetadataResolver.java:325) [opensaml-saml-impl-4.3.2.jar:?]
2025-01-09T15:37:45.8101236Z 	at org.opensaml.saml.metadata.resolver.impl.AbstractMetadataResolver.doInitialize(AbstractMetadataResolver.java:289) [opensaml-saml-impl-4.3.2.jar:?]
2025-01-09T15:37:45.8102369Z 	at net.shibboleth.utilities.java.support.component.AbstractInitializableComponent.initialize(AbstractInitializableComponent.java:65) [java-support-8.4.2.jar:?]
2025-01-09T15:37:45.8103672Z 	at com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator.lambda$createMetadataResolver$1(HTTPSamlAuthenticator.java:363) [opensearch-security-3.0.0.0-SNAPSHOT.jar:3.0.0.0-SNAPSHOT]
2025-01-09T15:37:45.8104582Z 	at java.base/java.security.AccessController.doPrivileged(AccessController.java:571) [?:?]
2025-01-09T15:37:45.8105479Z 	at com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator.createMetadataResolver(HTTPSamlAuthenticator.java:362) [opensearch-security-3.0.0.0-SNAPSHOT.jar:3.0.0.0-SNAPSHOT]
2025-01-09T15:37:45.8106587Z 	at com.amazon.dlic.auth.http.saml.HTTPSamlAuthenticator.<init>(HTTPSamlAuthenticator.java:134) [opensearch-security-3.0.0.0-SNAPSHOT.jar:3.0.0.0-SNAPSHOT]
2025-01-09T15:37:45.8107552Z 	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) ~[?:?]
2025-01-09T15:37:45.8108316Z 	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502) ~[?:?]
2025-01-09T15:37:45.8108897Z 	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486) ~[?:?]
...

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: Craig Perkins <[email protected]>
@cwperks cwperks mentioned this pull request Jan 30, 2025
3 tasks
@cwperks cwperks merged commit 90b2e77 into opensearch-project:main Jan 31, 2025
19 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 31, 2025
* Attempt to fix CI failures related to security plugin download

Signed-off-by: Craig Perkins <[email protected]>

* Make generic

Signed-off-by: Craig Perkins <[email protected]>

* Maven repository

Signed-off-by: Craig Perkins <[email protected]>

* debug

Signed-off-by: Craig Perkins <[email protected]>

* Add SNAPSHOT

Signed-off-by: Craig Perkins <[email protected]>

* Remove wildcard

Signed-off-by: Craig Perkins <[email protected]>

* Remove debug

Signed-off-by: Craig Perkins <[email protected]>

* Use IPv6

Signed-off-by: Craig Perkins <[email protected]>

* Wrap with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Remove cy.visit in before

Signed-off-by: Craig Perkins <[email protected]>

* Wrap with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Move up by 1

Signed-off-by: Craig Perkins <[email protected]>

* Pass basePath as arg

Signed-off-by: Craig Perkins <[email protected]>

* Wrap with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Intercept and replace ::1 with localhost

Signed-off-by: Craig Perkins <[email protected]>

* Intercept and replace ::1 with localhost

Signed-off-by: Craig Perkins <[email protected]>

* remove console.log

Signed-off-by: Craig Perkins <[email protected]>

* Surround with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* move cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Use req.on response

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
(cherry picked from commit 90b2e77)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 31, 2025
* Attempt to fix CI failures related to security plugin download

Signed-off-by: Craig Perkins <[email protected]>

* Make generic

Signed-off-by: Craig Perkins <[email protected]>

* Maven repository

Signed-off-by: Craig Perkins <[email protected]>

* debug

Signed-off-by: Craig Perkins <[email protected]>

* Add SNAPSHOT

Signed-off-by: Craig Perkins <[email protected]>

* Remove wildcard

Signed-off-by: Craig Perkins <[email protected]>

* Remove debug

Signed-off-by: Craig Perkins <[email protected]>

* Use IPv6

Signed-off-by: Craig Perkins <[email protected]>

* Wrap with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Remove cy.visit in before

Signed-off-by: Craig Perkins <[email protected]>

* Wrap with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Move up by 1

Signed-off-by: Craig Perkins <[email protected]>

* Pass basePath as arg

Signed-off-by: Craig Perkins <[email protected]>

* Wrap with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Intercept and replace ::1 with localhost

Signed-off-by: Craig Perkins <[email protected]>

* Intercept and replace ::1 with localhost

Signed-off-by: Craig Perkins <[email protected]>

* remove console.log

Signed-off-by: Craig Perkins <[email protected]>

* Surround with cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* move cy.origin

Signed-off-by: Craig Perkins <[email protected]>

* Use req.on response

Signed-off-by: Craig Perkins <[email protected]>

---------

Signed-off-by: Craig Perkins <[email protected]>
(cherry picked from commit 90b2e77)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
cwperks pushed a commit that referenced this pull request Jan 31, 2025
* Attempt to fix CI failures related to security plugin download



* Make generic



* Maven repository



* debug



* Add SNAPSHOT



* Remove wildcard



* Remove debug



* Use IPv6



* Wrap with cy.origin



* Remove cy.visit in before



* Wrap with cy.origin



* Move up by 1



* Pass basePath as arg



* Wrap with cy.origin



* Intercept and replace ::1 with localhost



* Intercept and replace ::1 with localhost



* remove console.log



* Surround with cy.origin



* move cy.origin



* Use req.on response



---------


(cherry picked from commit 90b2e77)

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
cwperks pushed a commit that referenced this pull request Jan 31, 2025
* Attempt to fix CI failures related to security plugin download



* Make generic



* Maven repository



* debug



* Add SNAPSHOT



* Remove wildcard



* Remove debug



* Use IPv6



* Wrap with cy.origin



* Remove cy.visit in before



* Wrap with cy.origin



* Move up by 1



* Pass basePath as arg



* Wrap with cy.origin



* Intercept and replace ::1 with localhost



* Intercept and replace ::1 with localhost



* remove console.log



* Surround with cy.origin



* move cy.origin



* Use req.on response



---------


(cherry picked from commit 90b2e77)

Signed-off-by: Craig Perkins <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x backport to 2.x branch backport 2.19
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[AUTOCUT] Distribution Build Failed for securityDashboards-3.0.0
2 participants