feat: add clickbench benchmark #1304
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
robert3005
reviewed
Nov 14, 2024
|
some benchmarks are failing, I'll fix them tomorrow |
lwwmanning
reviewed
Nov 15, 2024
also `storage_array` -> `storage_data` seems appropriate. --------- Co-authored-by: Robert Kruszewski <[email protected]>
robert3005
force-pushed
the
adamg/clickbench
branch
from
3fd8a38 to
910a11d
Compare
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Vortex bytes_at
| Benchmark suite | Current: 4814b0a | Previous: 1b62acf | Ratio |
|---|---|---|---|
bytes_at/array_data |
1029.2876587246208 ns (3.319326271482737) |
998.8755834607696 ns (1.125705602810683) |
1.03 |
bytes_at/array_view |
807.6846839944232 ns (0.9162343376843296) |
817.6418480591408 ns (0.870804783228948) |
0.99 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
DataFusion
| Benchmark suite | Current: 5c5a244 | Previous: 6696266 | Ratio |
|---|---|---|---|
arrow/planning |
809569.1556734097 ns (2762.5625421365257) |
804287.4003737285 ns (3060.038381783932) |
1.01 |
arrow/exec |
1757736.733816171 ns (6190.8734542913735) |
1768153.037634904 ns (5285.6973435455) |
0.99 |
vortex-pushdown-compressed/planning |
506305.9590412769 ns (4384.966998024873) |
506083.14675167593 ns (2309.68065084322) |
1.00 |
vortex-pushdown-compressed/exec |
2681393.09631579 ns (10849.058638158487) |
2665428.875263158 ns (11847.3367236841) |
1.01 |
vortex-pushdown-uncompressed/planning |
505146.53513867484 ns (1429.2338472832344) |
508341.47328858654 ns (1783.9559742663987) |
0.99 |
vortex-pushdown-uncompressed/exec |
1482192.3080452033 ns (6447.186318977969) |
1470004.3147268945 ns (4795.4555683872895) |
1.01 |
vortex-nopushdown-compressed/planning |
838373.6172758979 ns (3281.123078463308) |
841686.590321122 ns (2306.2376911277534) |
1.00 |
vortex-nopushdown-compressed/exec |
3483871.2699999996 ns (16965.911916666897) |
3807679.6830769223 ns (17064.216961538885) |
0.91 |
vortex-nopushdown-uncompressed/planning |
822480.7307428675 ns (2011.0270062295604) |
822935.650612076 ns (2457.635306605138) |
1.00 |
vortex-nopushdown-uncompressed/exec |
4889296.144545454 ns (8543.523977272678) |
5220411.030000001 ns (10408.77533749817) |
0.94 |
This comment was automatically generated by workflow using github-action-benchmark.
There was a problem hiding this comment.
Random Access
| Benchmark suite | Current: 5c5a244 | Previous: 6696266 | Ratio |
|---|---|---|---|
random-access/vortex-tokio-local-disk |
2289347.893478262 ns (27082.204798913328) |
2619887.8745000004 ns (13685.819999999367) |
0.87 |
random-access/vortex-local-fs |
2601281.826842105 ns (22047.510355263017) |
3009190.087647059 ns (12081.916294117458) |
0.86 |
random-access/parquet-tokio-local-disk |
217958041.9 ns (4641554.205833331) |
225727101.8333333 ns (2847408.4895833135) |
0.97 |
This comment was automatically generated by workflow using github-action-benchmark.
Ideally we would test every combination of features, but at the very least, we should verify that no features, default features, and all features pass clippy. The two main changes: 1. In vortex-io, we need a syntactically valid expressoin when neither compio nor tokio are available. 2. In vortex-array, we need both flatbuffers and flexbuffers for views to work. I removed the optional dependencies on these libraries. In theory, someone could go through our code base and properly feature flag array views, but that began to feel onerous as I did it.
AdamGS
force-pushed
the
adamg/clickbench
branch
from
d1f197d to
41a2e1f
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds a clickbench run to our datafusion benchmarks.
Some queries fail, see #1326.Running this is pretty resource intensive as the dataset is ~100m rows/15GB of parquet.