Change priority for scheduling reroute during timeout #16445
Conversation
Signed-off-by: Rishab Nahata <[email protected]>
|
❌ Gradle check result for 5e83a92: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
| "reroute after existing shards allocator timed out", | ||
| Priority.HIGH, | ||
| "reroute after existing shards allocator [R] timed out", | ||
| Priority.NORMAL, |
There was a problem hiding this comment.
Should we have a separate priority for primary vs replica?
There was a problem hiding this comment.
NORMAL also seems right for PSA. But during genuine issues in the cluster which can be identified with appropriate monitoring, we might need to raise it to HIGH. I will update the PR with a similar setting for ESA similar to BSA to raise reroute priority. Wdyt?
Updated |
Signed-off-by: Rishab Nahata <[email protected]>
imRishN
requested review from
anasalkouz,
andrross,
ashking94,
CEHENKLE,
dblock,
dbwiddis,
gbbafna,
jainankitk,
kotwanikunal,
linuxpi,
mch2,
msfroh,
nknize,
owaiskazi19,
reta,
Rishikesh1159,
sachinpkale,
saratvemulapalli,
shwetathareja,
sohami and
VachaShah
as code owners
|
❌ Gradle check result for 6a448d0: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
|
❌ Gradle check result for 825a983: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Rishab Nahata <[email protected]>
|
❌ Gradle check result for 5368e7f: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Rishab Nahata <[email protected]>
|
❌ Gradle check result for 2ba604d: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #16445 +/- ##
============================================
+ Coverage 72.24% 72.29% +0.05%
+ Complexity 65305 65300 -5
============================================
Files 5301 5301
Lines 303774 303798 +24
Branches 44016 44018 +2
============================================
+ Hits 219458 219632 +174
+ Misses 66272 66099 -173
- Partials 18044 18067 +23 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
| Setting.Property.NodeScope, | ||
| Setting.Property.Dynamic | ||
| ); | ||
|
|
There was a problem hiding this comment.
This logic seems redundant
There was a problem hiding this comment.
Do you mean to parse reroute priority?
There was a problem hiding this comment.
Exception thrown has a different message. This can be rearranged, but currently this state also looks cleaner
...c/main/java/org/opensearch/cluster/routing/allocation/allocator/BalancedShardsAllocator.java
Show resolved
Hide resolved
|
This PR is stalled because it has been open for 30 days with no activity. |
Signed-off-by: Rishab Nahata <[email protected]>
|
❌ Gradle check result for 7596691: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Rishab Nahata <[email protected]>
Signed-off-by: Rishab Nahata <[email protected]>
|
❌ Gradle check result for bf24a5b: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Description
This PR updates the priority of scheduling reroute when timed out from HIGH to NORMAL. This is because consistent HIGH reroutes might starve NORMAL priority tasks. And moreover, NORMAL is right for reasonable clusters. For clusters in messed up state which is causing NORMAL priority tasks to starve, we add a new dynamic cluster setting to raise the priority of reroute task to allocate shards in such scenarios.
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
[ ] Functionality includes testing.[ ] API changes companion pull request created, if applicable.[ ] Public documentation issue/PR created, if applicable.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.