Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

very slow i/o on arc #368

Open
drtobybrown opened this issue Jul 20, 2022 · 0 comments
Open

very slow i/o on arc #368

drtobybrown opened this issue Jul 20, 2022 · 0 comments

Comments

@drtobybrown
Copy link
Collaborator

Read/writing data on arc is ~30x slower than on MacBook Pro

MacBook Pro (14-inch, 2021)

» wget https://gist.githubusercontent.com/Jimilian/29a11c9ba799ebd77abf129e7339422e/raw/bb9a95e863a5d8e86fc36a1c829741b880d63ce4/perf_test_serializators.py
» python perf_test_serializators.py 
iterations= 100000
data= 'avzvasdklfjhaskldjfhkweljrqlkjb*@&$Y)(!#&$G@#lkjabfsdflb(*!G@#$(GKLJBmnz,bv(PGDFLKJ'
==== DUMP ====
Pickle:
>> 0.02323204200000001
Json: 2.0.9
>> 0.052325416999999985
Marshal: 4
>> 0.015486083999999983
Msgpack: (1, 0, 4)
>> 0.020649415999999976

==== LOAD ====
Pickle:
>> 0.014170584000000042
Json: 2.0.9
>> 0.08166637500000001
Marshal: 4
>> 0.00776574999999996
Msgpack: (1, 0, 4)
>> 0.009213915999999989

science platform arc/

...
==== DUMP ====
Pickle:
>> 0.20738047594204545
Json: 2.0.9
>> 0.8866461659781635
Marshal: 4
>> 0.30416943319141865
Msgpack: (1, 0, 3)
>> 0.4073137487284839

==== LOAD ====
Pickle:
>> 0.39129116013646126
Json: 2.0.9
>> 1.80569237889722
Marshal: 4
>> 0.2898394321091473
Msgpack: (1, 0, 3)
>> 0.2016866127960384
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant