Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⬆️ update multiprocess(ing) #91

Closed
wants to merge 9 commits into from
Closed

⬆️ update multiprocess(ing) #91

wants to merge 9 commits into from

Conversation

jvdd
Copy link
Member

@jvdd jvdd commented Jan 28, 2023

Should resolve #90

  • update dill & multiprocess dependencies
  • enable multiprocessing on Windows

TODO:

  • look into consequences of upgrading dill dependencies

I got it to work on Windows - the following observations:

  • dill.settings["recurse"] = True is not necessary
  • we cannot pickle (dill) generators pickle iterators and generators uqfoundation/dill#10
    => code needs some updates (currently done quite hacky)
  • multiprocessing on Windows often works slower than non-multiprocessing. Largest improvement that I noticed was like 5%...

As for now I will just close this PR and keep the implementation as it was

@codecov-commenter
Copy link

codecov-commenter commented Jan 28, 2023

Codecov Report

Merging #91 (e8bdaef) into main (8ac5086) will increase coverage by 0.08%.
The diff coverage is 98.68%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             main      #91      +/-   ##
==========================================
+ Coverage   97.91%   97.99%   +0.08%     
==========================================
  Files          23       23              
  Lines        1245     1246       +1     
==========================================
+ Hits         1219     1221       +2     
+ Misses         26       25       -1     
Impacted Files Coverage Δ
tsflex/processing/utils.py 74.19% <50.00%> (ø)
tsflex/__init__.py 100.00% <100.00%> (ø)
tsflex/chunking/__init__.py 100.00% <100.00%> (ø)
tsflex/chunking/chunking.py 98.07% <100.00%> (ø)
tsflex/features/feature.py 97.91% <100.00%> (ø)
tsflex/features/feature_collection.py 100.00% <100.00%> (+0.47%) ⬆️
tsflex/features/function_wrapper.py 100.00% <100.00%> (ø)
tsflex/features/integrations.py 98.48% <100.00%> (ø)
tsflex/features/logger.py 100.00% <100.00%> (ø)
tsflex/features/segmenter/strided_rolling.py 98.29% <100.00%> (ø)
... and 10 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

⬆️ update dill & multiprocess dependencies
2 participants