-
Notifications
You must be signed in to change notification settings - Fork 195
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add extra check in fix_job_kwargs #2829
Conversation
src/spikeinterface/core/job_tools.py
Outdated
@@ -87,9 +87,14 @@ def fix_job_kwargs(runtime_job_kwargs): | |||
n_jobs = job_kwargs["n_jobs"] | |||
assert isinstance(n_jobs, (float, np.integer, int)) | |||
if isinstance(n_jobs, float): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if isinstance(n_jobs, float): | |
if isinstance(n_jobs, float) and n_jobs <= 1: |
@zm711 maybe this is better?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we would need to add another:
else:
n_jobs = int(n_jobs)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So flip the if-else so the more common path is triggered first?
if isinstance(n_jobs, float) and n_jobs <= 1:
int(n_jobs * os.cpu_counts())
else:
int(n_jobs)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see you're saying fuse all into if-elif-else!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let me know what you think of this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep looks great!
I personally think that we should raise an error if someone sets 0 jobs rather than fix it for them.... Should I change that test or revert raising the error? |
The old behavior was to set |
For functions that take in an
n_jobs
argument they all runfix_job_kwargs
, but if the user gives something like 4.0 to mean 4 jobs they end up with (if they have 4 cores)int(4.0 * 4.0) = 16
So instead we should cast 4.0 -> 4 and just make it 4 jobs no?
What do you think @alejoe91 ?