Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to disable job tracker update when a compaction job is committed #4132

Open
patchwork01 opened this issue Jan 24, 2025 · 0 comments
Open

Comments

@patchwork01
Copy link
Collaborator

patchwork01 commented Jan 24, 2025

Background

Split from:

Description

Given that we're batching up updates to the state store for a large number of compactions, we'd like to find out how many jobs the system can handle at once. This means putting more compactions through the system than the state store commit lambda can apply to the job tracker.

We'd like an option to disable job tracker updates for compactions in the state store commit lambda. We don't need to care how this affects the reporting since we will only use this option for testing.

Analysis

The job tracker update happens in StateStoreCommitter.

@patchwork01 patchwork01 added this to the 0.28.0 milestone Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant