Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(execution): Exclude execution retrieval for Disabled Front50 pipelines #4819

Merged

Conversation

christosarvanitis
Copy link
Member

@christosarvanitis christosarvanitis commented Dec 19, 2024

Adding an opt-in feature to exclude Execution Retrieval for Disabled Front50 pipelines. This can be configured via:

tasks:
   controller:
      excludeExecutionsOfDisabledPipelines: true #(defaults to false) 

When enabled Orca calls the Front50 with a query parameter enabledPipelines=true. Front50 responds only with the enabled pipelines of the application.

This feature aims to ease the load on the execution retrieval for applications that have a lot of pipelines and for historical reasons users dont want to delete the obsolete pipelines that are disabled.

Additionally an agent is implemented to check for unused/unexecuted pipelines per application for the past X days and disable them in Front50. This is only applicable for SQL execution repository and can be configured via:

pollers:
  unused-pipelines-disable:
    enabled: false|true #default value false
    intervalSec: 3600 #default value 3600
    thresholdDays: 365 #default value 365
    dryRun: false|true #default value true

Related Front50 change spinnaker/front50#1520

@christosarvanitis christosarvanitis force-pushed the feat/excludeExecutionForDisabledPipelines branch from 5c86e68 to f849250 Compare December 30, 2024 15:49
@christosarvanitis christosarvanitis force-pushed the feat/excludeExecutionForDisabledPipelines branch from f849250 to 086a55d Compare December 30, 2024 16:19
@christosarvanitis christosarvanitis marked this pull request as ready for review December 30, 2024 16:39
@christosarvanitis christosarvanitis force-pushed the feat/excludeExecutionForDisabledPipelines branch from e5a4857 to 412524c Compare January 3, 2025 11:58
criteria: ExecutionCriteria
): List<String> {
var baseQueryPredicate = field("application").eq(application)
var table = if (jooq.dialect() == SQLDialect.MYSQL) PIPELINE.tableName.forceIndex("pipeline_application_idx")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a rule, try to avoid force indexes - let the database optimize as needed as if new indexes come online or old ones get removed this would require work here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following the optimised retrieval from this PR #4804 I can revert to a simple query. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please! Simple == better on these let the DB do it's thing... particularly if in future indexes change.

@christosarvanitis christosarvanitis force-pushed the feat/excludeExecutionForDisabledPipelines branch 3 times, most recently from 1a280e1 to c3e3f22 Compare January 17, 2025 14:11
@christosarvanitis christosarvanitis force-pushed the feat/excludeExecutionForDisabledPipelines branch from 29d8ef1 to 94eb5b8 Compare January 23, 2025 09:08
@jasonmcintosh jasonmcintosh added the ready to merge Approved and ready for merge label Jan 24, 2025
@mergify mergify bot added the auto merged Merged automatically by a bot label Jan 24, 2025
@mergify mergify bot merged commit 33097a3 into spinnaker:master Jan 24, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto merged Merged automatically by a bot ready to merge Approved and ready for merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants