Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable read only DB tests on windows #4200

Merged
merged 1 commit into from
Sep 16, 2024
Merged

Conversation

benjaminwinger
Copy link
Collaborator

Support for this was implemented a while ago (I don't recall exactly when), but evidently these tests were missed.

Copy link

codecov bot commented Sep 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.75%. Comparing base (796f70d) to head (3fb4af2).
Report is 20 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4200      +/-   ##
==========================================
+ Coverage   86.65%   86.75%   +0.10%     
==========================================
  Files        1327     1327              
  Lines       51365    51581     +216     
  Branches     6895     6948      +53     
==========================================
+ Hits        44511    44751     +240     
+ Misses       6682     6658      -24     
  Partials      172      172              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

github-actions bot commented Sep 5, 2024

Benchmark Result

Master commit hash: b40bee1042625fa8697829ca0f0b3c096ac4e92d
Branch commit hash: 74ee2bb63f615154e650dac7cbc2799bce198e3c

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 674.61 683.56 -8.95 (-1.31%)
aggregation q28 11451.67 11170.86 280.81 (2.51%)
filter q14 153.07 151.38 1.69 (1.12%)
filter q15 153.39 152.41 0.99 (0.65%)
filter q16 335.21 333.00 2.21 (0.66%)
filter q17 473.89 475.44 -1.55 (-0.33%)
filter q18 1952.23 1964.79 -12.56 (-0.64%)
fixed_size_expr_evaluator q07 568.55 567.03 1.51 (0.27%)
fixed_size_expr_evaluator q08 781.66 772.76 8.91 (1.15%)
fixed_size_expr_evaluator q09 780.75 774.51 6.24 (0.81%)
fixed_size_expr_evaluator q10 265.14 265.50 -0.36 (-0.14%)
fixed_size_expr_evaluator q11 259.72 262.00 -2.29 (-0.87%)
fixed_size_expr_evaluator q12 257.73 259.13 -1.40 (-0.54%)
fixed_size_expr_evaluator q13 1491.87 1506.77 -14.90 (-0.99%)
fixed_size_seq_scan q23 143.36 142.05 1.32 (0.93%)
join q31 12.55 13.14 -0.59 (-4.50%)
ldbc_snb_ic q35 758.74 742.67 16.07 (2.16%)
ldbc_snb_ic q36 27.69 45.63 -17.94 (-39.32%)
ldbc_snb_is q32 8.07 8.36 -0.29 (-3.42%)
ldbc_snb_is q33 14.89 15.65 -0.77 (-4.89%)
ldbc_snb_is q34 7.82 7.44 0.38 (5.12%)
multi-rel multi-rel-large-scan 3423.74 2816.85 606.89 (21.54%)
multi-rel multi-rel-lookup 57.14 57.71 -0.57 (-0.98%)
multi-rel multi-rel-small-scan 50.69 49.48 1.21 (2.44%)
order_by q25 157.65 154.48 3.17 (2.05%)
order_by q26 478.36 480.41 -2.05 (-0.43%)
order_by q27 1416.51 1413.93 2.58 (0.18%)
scan_after_filter q01 198.81 199.53 -0.72 (-0.36%)
scan_after_filter q02 187.41 188.47 -1.06 (-0.56%)
shortest_path_ldbc100 q39 74.22 75.85 -1.63 (-2.15%)
var_size_expr_evaluator q03 2070.49 2066.44 4.05 (0.20%)
var_size_expr_evaluator q04 2233.77 2244.78 -11.01 (-0.49%)
var_size_expr_evaluator q05 2631.33 2626.48 4.85 (0.18%)
var_size_expr_evaluator q06 1467.67 1475.62 -7.95 (-0.54%)
var_size_seq_scan q19 1481.96 1475.82 6.14 (0.42%)
var_size_seq_scan q20 3293.83 3184.61 109.22 (3.43%)
var_size_seq_scan q21 2512.44 2445.74 66.70 (2.73%)
var_size_seq_scan q22 133.04 132.95 0.09 (0.06%)

Copy link

github-actions bot commented Sep 6, 2024

Benchmark Result

Master commit hash: 3c86dddb8801004cac6d4c1d9f465ec0b08bb6fd
Branch commit hash: 147aaf7075d2b7f891e9d991ebc410aaf3743b59

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 670.67 672.67 -2.00 (-0.30%)
aggregation q28 11887.45 11158.15 729.30 (6.54%)
filter q14 155.28 151.95 3.33 (2.19%)
filter q15 154.15 150.22 3.93 (2.61%)
filter q16 333.91 332.00 1.91 (0.58%)
filter q17 475.81 474.42 1.39 (0.29%)
filter q18 1965.03 1954.70 10.33 (0.53%)
fixed_size_expr_evaluator q07 563.27 566.07 -2.81 (-0.50%)
fixed_size_expr_evaluator q08 772.61 781.87 -9.26 (-1.18%)
fixed_size_expr_evaluator q09 773.86 776.12 -2.26 (-0.29%)
fixed_size_expr_evaluator q10 264.68 264.68 0.00 (0.00%)
fixed_size_expr_evaluator q11 259.65 259.72 -0.07 (-0.03%)
fixed_size_expr_evaluator q12 258.88 258.66 0.22 (0.08%)
fixed_size_expr_evaluator q13 1491.72 1494.35 -2.63 (-0.18%)
fixed_size_seq_scan q23 143.79 143.10 0.69 (0.48%)
join q31 12.15 12.99 -0.85 (-6.53%)
ldbc_snb_ic q35 731.03 774.43 -43.40 (-5.60%)
ldbc_snb_ic q36 42.37 49.61 -7.24 (-14.60%)
ldbc_snb_is q32 8.31 9.02 -0.72 (-7.93%)
ldbc_snb_is q33 17.70 18.53 -0.83 (-4.46%)
ldbc_snb_is q34 7.99 8.59 -0.60 (-6.93%)
multi-rel multi-rel-large-scan 2762.84 2827.68 -64.84 (-2.29%)
multi-rel multi-rel-lookup 70.50 53.59 16.91 (31.55%)
multi-rel multi-rel-small-scan 54.55 45.95 8.60 (18.71%)
order_by q25 157.22 158.21 -0.98 (-0.62%)
order_by q26 475.71 477.26 -1.55 (-0.32%)
order_by q27 1413.36 1423.17 -9.81 (-0.69%)
scan_after_filter q01 199.83 200.00 -0.17 (-0.09%)
scan_after_filter q02 187.94 187.56 0.38 (0.20%)
shortest_path_ldbc100 q39 72.83 73.21 -0.38 (-0.52%)
var_size_expr_evaluator q03 2065.36 2077.22 -11.86 (-0.57%)
var_size_expr_evaluator q04 2235.96 2253.37 -17.41 (-0.77%)
var_size_expr_evaluator q05 2628.53 2621.55 6.98 (0.27%)
var_size_expr_evaluator q06 1465.32 1476.22 -10.90 (-0.74%)
var_size_seq_scan q19 1477.22 1473.99 3.23 (0.22%)
var_size_seq_scan q20 3207.49 3254.18 -46.70 (-1.43%)
var_size_seq_scan q21 2429.48 2435.71 -6.23 (-0.26%)
var_size_seq_scan q22 131.79 131.87 -0.08 (-0.06%)

Exception,
match=r"RuntimeError: IO exception: Could not set lock on file",
):
open_database_on_subprocess(db_path, build_dir)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did some tests with the shell, and as far as I can tell on windows you can open the same database in read-write mode, operate on the files, and the files get clobbered by whichever writes last. The locks don't seem to be working properly. It works as expected if you open one in read-only mode though.
The C++ test that does something similar is being skipped, presumably because in addition to windows not supporting this initially, it also wasn't written to work with msvc (among other things it uses fork to create the process, which doesn't exist on windows).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, the lock is not supported on windows yet. #921

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess not. I'd seen locking code from #2089, but it doesn't appear to work.
I have a suspicion that the issue is that it locks a region of the file, not the entire file, so locking write access to a zero-length region (as our code currently does) doesn't actually do anything. But maybe it would work if we lock even a single byte instead.

@benjaminwinger benjaminwinger force-pushed the windows-read-only branch 2 times, most recently from 559b2cd to ba4822e Compare September 10, 2024 09:49
Copy link

Benchmark Result

Master commit hash: 2560eabe74cf743d4166b1d164a448773bc923bb
Branch commit hash: 743248963540f129e61e3efebd4f44504c38bf3e

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 678.77 680.90 -2.14 (-0.31%)
aggregation q28 11395.85 11849.00 -453.14 (-3.82%)
filter q14 159.97 158.27 1.70 (1.07%)
filter q15 161.76 160.33 1.43 (0.89%)
filter q16 347.32 349.38 -2.06 (-0.59%)
filter q17 483.35 479.72 3.63 (0.76%)
filter q18 1944.99 1920.05 24.94 (1.30%)
fixed_size_expr_evaluator q07 574.59 573.55 1.04 (0.18%)
fixed_size_expr_evaluator q08 791.60 786.80 4.80 (0.61%)
fixed_size_expr_evaluator q09 797.12 789.55 7.57 (0.96%)
fixed_size_expr_evaluator q10 277.50 278.51 -1.01 (-0.36%)
fixed_size_expr_evaluator q11 270.97 273.11 -2.14 (-0.78%)
fixed_size_expr_evaluator q12 270.19 269.92 0.27 (0.10%)
fixed_size_expr_evaluator q13 1503.41 1500.27 3.15 (0.21%)
fixed_size_seq_scan q23 154.14 156.33 -2.19 (-1.40%)
join q31 11.80 12.30 -0.49 (-4.01%)
ldbc_snb_ic q35 771.74 787.80 -16.06 (-2.04%)
ldbc_snb_ic q36 54.44 46.95 7.49 (15.96%)
ldbc_snb_is q32 8.52 8.84 -0.31 (-3.55%)
ldbc_snb_is q33 16.26 16.21 0.05 (0.33%)
ldbc_snb_is q34 7.99 7.86 0.12 (1.59%)
multi-rel multi-rel-large-scan 2929.65 2957.07 -27.42 (-0.93%)
multi-rel multi-rel-lookup 64.74 75.68 -10.94 (-14.45%)
multi-rel multi-rel-small-scan 47.07 54.92 -7.85 (-14.29%)
order_by q25 166.06 164.87 1.20 (0.73%)
order_by q26 487.74 486.67 1.07 (0.22%)
order_by q27 1439.78 1439.45 0.33 (0.02%)
scan_after_filter q01 207.97 208.11 -0.14 (-0.07%)
scan_after_filter q02 195.52 194.63 0.89 (0.46%)
shortest_path_ldbc100 q37 3396.94 4827.00 -1430.06 (-29.63%)
shortest_path_ldbc100 q39 110.15 112.25 -2.11 (-1.88%)
shortest_path_ldbc100 q40 67.11 71.75 -4.64 (-6.46%)
var_size_expr_evaluator q03 2094.83 2105.17 -10.34 (-0.49%)
var_size_expr_evaluator q04 2240.69 2265.91 -25.22 (-1.11%)
var_size_expr_evaluator q05 2624.92 2624.75 0.17 (0.01%)
var_size_expr_evaluator q06 1401.84 1403.82 -1.98 (-0.14%)
var_size_seq_scan q19 1504.24 1498.35 5.89 (0.39%)
var_size_seq_scan q20 3261.36 3246.07 15.29 (0.47%)
var_size_seq_scan q21 2496.50 2517.15 -20.65 (-0.82%)
var_size_seq_scan q22 136.87 136.51 0.36 (0.26%)

@benjaminwinger benjaminwinger force-pushed the windows-read-only branch 2 times, most recently from 6e8a160 to d19b19d Compare September 11, 2024 09:21
Copy link

Benchmark Result

Master commit hash: 796f70df4141b872a179f3aa35be0ea412f204e7
Branch commit hash: 2dc07056f2f697724ecb627650c2c7d43f8afcf6

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 678.77 679.38 -0.61 (-0.09%)
aggregation q28 11673.41 11147.20 526.21 (4.72%)
filter q14 161.02 158.69 2.33 (1.47%)
filter q15 158.88 158.55 0.33 (0.21%)
filter q16 334.81 334.56 0.26 (0.08%)
filter q17 479.89 480.74 -0.84 (-0.18%)
filter q18 1930.42 1922.99 7.43 (0.39%)
fixed_size_expr_evaluator q07 569.39 577.52 -8.14 (-1.41%)
fixed_size_expr_evaluator q08 785.05 785.53 -0.49 (-0.06%)
fixed_size_expr_evaluator q09 787.36 791.78 -4.42 (-0.56%)
fixed_size_expr_evaluator q10 273.76 275.42 -1.65 (-0.60%)
fixed_size_expr_evaluator q11 266.87 269.42 -2.55 (-0.95%)
fixed_size_expr_evaluator q12 265.99 267.08 -1.09 (-0.41%)
fixed_size_expr_evaluator q13 1513.78 1509.35 4.43 (0.29%)
fixed_size_seq_scan q23 147.48 149.00 -1.52 (-1.02%)
join q31 12.25 11.39 0.86 (7.56%)
ldbc_snb_ic q35 743.10 876.99 -133.89 (-15.27%)
ldbc_snb_ic q36 46.24 52.32 -6.08 (-11.61%)
ldbc_snb_is q32 9.43 8.37 1.06 (12.64%)
ldbc_snb_is q33 17.30 13.59 3.71 (27.26%)
ldbc_snb_is q34 8.66 7.99 0.67 (8.38%)
multi-rel multi-rel-large-scan 2791.99 2721.99 70.00 (2.57%)
multi-rel multi-rel-lookup 74.54 65.47 9.07 (13.85%)
multi-rel multi-rel-small-scan 59.73 48.42 11.30 (23.35%)
order_by q25 159.64 163.08 -3.44 (-2.11%)
order_by q26 489.54 500.70 -11.16 (-2.23%)
order_by q27 1440.27 1444.44 -4.17 (-0.29%)
scan_after_filter q01 204.03 204.96 -0.93 (-0.45%)
scan_after_filter q02 193.17 194.82 -1.64 (-0.84%)
shortest_path_ldbc100 q37 3464.65 3724.36 -259.71 (-6.97%)
shortest_path_ldbc100 q39 119.36 111.39 7.97 (7.15%)
shortest_path_ldbc100 q40 71.88 68.41 3.47 (5.08%)
var_size_expr_evaluator q03 2097.13 2104.69 -7.56 (-0.36%)
var_size_expr_evaluator q04 2259.59 2264.26 -4.67 (-0.21%)
var_size_expr_evaluator q05 2581.28 2580.28 0.99 (0.04%)
var_size_expr_evaluator q06 1421.19 1430.30 -9.11 (-0.64%)
var_size_seq_scan q19 1510.85 1512.29 -1.45 (-0.10%)
var_size_seq_scan q20 3260.66 3259.98 0.69 (0.02%)
var_size_seq_scan q21 2491.33 2481.59 9.74 (0.39%)
var_size_seq_scan q22 137.90 137.09 0.81 (0.59%)

Copy link

Benchmark Result

Master commit hash: 1437af983fbc07e93b864050b812142043fe1c7e
Branch commit hash: e64c0ed42a99c10cf2d556c146d4b811b824c763

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 678.53 678.26 0.27 (0.04%)
aggregation q28 12363.01 12635.63 -272.62 (-2.16%)
copy node-Comment 76973.76 78319.44 -1345.68 (-1.72%)
copy node-Forum 4986.40 5143.75 -157.35 (-3.06%)
copy node-Organisation 1180.23 1518.04 -337.81 (-22.25%)
copy node-Person 1764.89 2678.62 -913.73 (-34.11%)
copy node-Place 1104.26 1479.79 -375.53 (-25.38%)
copy node-Post 32675.06 24263.30 8411.76 (34.67%)
copy node-Tag 1195.09 1535.63 -340.54 (-22.18%)
copy node-Tagclass 1103.33 457.83 645.50 (140.99%)
copy rel-comment-hasCreator 61712.33 58233.36 3478.97 (5.97%)
copy rel-comment-hasTag 77642.22 81410.29 -3768.07 (-4.63%)
copy rel-comment-isLocatedIn 62886.41 68684.59 -5798.18 (-8.44%)
copy rel-containerOf 17359.09 18373.41 -1014.32 (-5.52%)
copy rel-forum-hasTag 3744.69 4049.42 -304.73 (-7.53%)
copy rel-hasInterest 2470.53 2617.00 -146.47 (-5.60%)
copy rel-hasMember 53000.68 55376.06 -2375.38 (-4.29%)
copy rel-hasModerator 1557.64 1746.31 -188.67 (-10.80%)
copy rel-hasType 180.44 431.54 -251.10 (-58.19%)
copy rel-isPartOf 209.57 423.55 -213.98 (-50.52%)
copy rel-isSubclassOf 202.80 499.49 -296.69 (-59.40%)
copy rel-knows 6088.15 6001.64 86.51 (1.44%)
copy rel-likes-comment 106151.95 86626.88 19525.07 (22.54%)
copy rel-likes-post 31691.49 32673.51 -982.02 (-3.01%)
copy rel-organisation-isLocatedIn 189.03 729.56 -540.53 (-74.09%)
copy rel-person-isLocatedIn 377.19 750.61 -373.42 (-49.75%)
copy rel-post-hasCreator 15890.56 17201.94 -1311.38 (-7.62%)
copy rel-post-hasTag 21289.63 22346.19 -1056.56 (-4.73%)
copy rel-post-isLocatedIn 17539.31 17406.03 133.28 (0.77%)
copy rel-replyOf-comment 56579.33 73666.53 -17087.20 (-23.20%)
copy rel-replyOf-post 49216.59 48218.21 998.38 (2.07%)
copy rel-studyAt 389.41 881.86 -492.45 (-55.84%)
copy rel-workAt 655.48 1036.75 -381.27 (-36.78%)
filter q14 158.63 151.35 7.28 (4.81%)
filter q15 160.25 148.48 11.77 (7.93%)
filter q16 334.31 333.71 0.60 (0.18%)
filter q17 480.60 472.32 8.29 (1.75%)
filter q18 1913.07 1953.82 -40.75 (-2.09%)
fixed_size_expr_evaluator q07 580.29 564.09 16.21 (2.87%)
fixed_size_expr_evaluator q08 797.13 778.07 19.06 (2.45%)
fixed_size_expr_evaluator q09 802.47 777.33 25.14 (3.23%)
fixed_size_expr_evaluator q10 273.11 263.94 9.17 (3.47%)
fixed_size_expr_evaluator q11 268.43 259.03 9.40 (3.63%)
fixed_size_expr_evaluator q12 265.30 258.35 6.95 (2.69%)
fixed_size_expr_evaluator q13 1501.95 1491.55 10.39 (0.70%)
fixed_size_seq_scan q23 152.83 142.92 9.91 (6.94%)
join q31 12.10 11.90 0.19 (1.62%)
ldbc_snb_ic q35 756.59 756.94 -0.35 (-0.05%)
ldbc_snb_ic q36 43.86 51.16 -7.31 (-14.28%)
ldbc_snb_is q32 9.71 9.17 0.54 (5.92%)
ldbc_snb_is q33 16.10 17.60 -1.51 (-8.56%)
ldbc_snb_is q34 8.23 8.96 -0.73 (-8.15%)
multi-rel multi-rel-large-scan 2730.37 2729.22 1.15 (0.04%)
multi-rel multi-rel-lookup 60.96 42.81 18.15 (42.40%)
multi-rel multi-rel-small-scan 58.15 61.02 -2.87 (-4.71%)
order_by q25 165.57 155.87 9.69 (6.22%)
order_by q26 494.12 469.71 24.41 (5.20%)
order_by q27 1444.79 1422.96 21.84 (1.53%)
scan_after_filter q01 204.81 198.72 6.09 (3.06%)
scan_after_filter q02 192.44 187.12 5.32 (2.84%)
shortest_path_ldbc100 q37 3431.48 N/A N/A
shortest_path_ldbc100 q39 76.07 86.99 -10.92 (-12.55%)
shortest_path_ldbc100 q40 79.04 N/A N/A
var_size_expr_evaluator q03 2106.08 2070.11 35.97 (1.74%)
var_size_expr_evaluator q04 2283.66 2243.36 40.31 (1.80%)
var_size_expr_evaluator q05 2586.28 2695.35 -109.07 (-4.05%)
var_size_expr_evaluator q06 1421.56 1347.06 74.50 (5.53%)
var_size_seq_scan q19 1510.27 1481.76 28.51 (1.92%)
var_size_seq_scan q20 3185.11 3037.70 147.42 (4.85%)
var_size_seq_scan q21 2492.22 2397.21 95.02 (3.96%)
var_size_seq_scan q22 137.39 135.37 2.02 (1.49%)

Copy link

Benchmark Result

Master commit hash: ac72a4e3199da3850b22803654fe0726a84f9f60
Branch commit hash: 3c51d4afc12f625ed2e485c768fab00eaf9f5701

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 654.81 656.11 -1.30 (-0.20%)
aggregation q28 11881.39 11561.00 320.39 (2.77%)
filter q14 134.94 136.48 -1.55 (-1.13%)
filter q15 137.37 137.36 0.01 (0.01%)
filter q16 316.79 317.90 -1.10 (-0.35%)
filter q17 455.75 459.50 -3.74 (-0.81%)
filter q18 1945.05 1952.04 -6.99 (-0.36%)
fixed_size_expr_evaluator q07 549.76 552.06 -2.30 (-0.42%)
fixed_size_expr_evaluator q08 755.52 757.07 -1.55 (-0.20%)
fixed_size_expr_evaluator q09 755.91 762.66 -6.76 (-0.89%)
fixed_size_expr_evaluator q10 248.46 250.11 -1.65 (-0.66%)
fixed_size_expr_evaluator q11 243.30 243.13 0.18 (0.07%)
fixed_size_expr_evaluator q12 242.66 241.75 0.91 (0.38%)
fixed_size_expr_evaluator q13 1477.76 1476.20 1.55 (0.11%)
fixed_size_seq_scan q23 126.00 124.09 1.91 (1.54%)
join q31 11.78 12.09 -0.31 (-2.57%)
ldbc_snb_ic q35 784.36 784.02 0.34 (0.04%)
ldbc_snb_ic q36 50.49 49.25 1.23 (2.51%)
ldbc_snb_is q32 5.03 9.16 -4.13 (-45.12%)
ldbc_snb_is q33 16.93 18.94 -2.02 (-10.64%)
ldbc_snb_is q34 5.57 8.38 -2.81 (-33.51%)
multi-rel multi-rel-large-scan 2968.32 2960.52 7.80 (0.26%)
multi-rel multi-rel-lookup 44.24 49.49 -5.25 (-10.60%)
multi-rel multi-rel-small-scan 47.15 55.91 -8.76 (-15.66%)
order_by q25 143.18 138.84 4.34 (3.13%)
order_by q26 460.06 459.56 0.49 (0.11%)
order_by q27 1431.30 1459.23 -27.93 (-1.91%)
scan_after_filter q01 184.96 182.70 2.25 (1.23%)
scan_after_filter q02 172.00 170.22 1.78 (1.04%)
shortest_path_ldbc100 q37 3366.87 3774.94 -408.07 (-10.81%)
shortest_path_ldbc100 q39 52.46 52.91 -0.46 (-0.86%)
shortest_path_ldbc100 q40 82.99 71.54 11.45 (16.01%)
var_size_expr_evaluator q03 2124.81 2082.38 42.43 (2.04%)
var_size_expr_evaluator q04 2253.65 2260.47 -6.82 (-0.30%)
var_size_expr_evaluator q05 2672.55 2659.54 13.00 (0.49%)
var_size_expr_evaluator q06 1391.43 1390.41 1.02 (0.07%)
var_size_seq_scan q19 1494.51 1494.14 0.36 (0.02%)
var_size_seq_scan q20 3179.35 3159.18 20.17 (0.64%)
var_size_seq_scan q21 2457.30 2442.53 14.77 (0.60%)
var_size_seq_scan q22 130.76 132.43 -1.67 (-1.26%)

@benjaminwinger benjaminwinger force-pushed the windows-read-only branch 3 times, most recently from 5beaf88 to ce4de37 Compare September 12, 2024 10:48
Copy link

Benchmark Result

Master commit hash: 9eceda0ad3e50053b4410c665753af59228efc63
Branch commit hash: 43384617ad8c0a064544ca5ce5eb971aabb55763

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 658.41 654.67 3.74 (0.57%)
aggregation q28 11785.49 12206.54 -421.06 (-3.45%)
filter q14 134.25 135.51 -1.26 (-0.93%)
filter q15 136.26 132.54 3.72 (2.81%)
filter q16 316.84 316.98 -0.14 (-0.04%)
filter q17 458.22 458.18 0.05 (0.01%)
filter q18 1983.79 1943.55 40.23 (2.07%)
fixed_size_expr_evaluator q07 548.11 546.74 1.37 (0.25%)
fixed_size_expr_evaluator q08 763.39 757.63 5.76 (0.76%)
fixed_size_expr_evaluator q09 764.58 758.37 6.22 (0.82%)
fixed_size_expr_evaluator q10 249.55 249.01 0.54 (0.22%)
fixed_size_expr_evaluator q11 243.93 244.72 -0.78 (-0.32%)
fixed_size_expr_evaluator q12 241.93 242.84 -0.91 (-0.37%)
fixed_size_expr_evaluator q13 1479.97 1484.92 -4.95 (-0.33%)
fixed_size_seq_scan q23 126.17 126.70 -0.53 (-0.42%)
join q31 11.55 9.54 2.01 (21.01%)
ldbc_snb_ic q35 788.96 803.41 -14.45 (-1.80%)
ldbc_snb_ic q36 47.44 50.68 -3.24 (-6.40%)
ldbc_snb_is q32 9.26 8.93 0.33 (3.71%)
ldbc_snb_is q33 18.79 18.44 0.35 (1.89%)
ldbc_snb_is q34 9.24 8.32 0.92 (11.00%)
multi-rel multi-rel-large-scan 2966.17 3550.93 -584.76 (-16.47%)
multi-rel multi-rel-lookup 34.58 42.47 -7.88 (-18.56%)
multi-rel multi-rel-small-scan 52.73 49.30 3.43 (6.96%)
order_by q25 137.41 139.58 -2.17 (-1.56%)
order_by q26 459.10 458.34 0.75 (0.16%)
order_by q27 1421.83 1419.30 2.53 (0.18%)
scan_after_filter q01 182.74 181.30 1.44 (0.79%)
scan_after_filter q02 171.45 169.59 1.86 (1.10%)
shortest_path_ldbc100 q37 3527.42 3396.88 130.54 (3.84%)
shortest_path_ldbc100 q39 53.08 56.75 -3.67 (-6.47%)
shortest_path_ldbc100 q40 66.11 83.07 -16.96 (-20.41%)
var_size_expr_evaluator q03 2070.44 2084.99 -14.55 (-0.70%)
var_size_expr_evaluator q04 2253.40 2255.06 -1.66 (-0.07%)
var_size_expr_evaluator q05 2636.28 2630.77 5.51 (0.21%)
var_size_expr_evaluator q06 1388.90 1393.90 -5.01 (-0.36%)
var_size_seq_scan q19 1481.25 1482.05 -0.81 (-0.05%)
var_size_seq_scan q20 3195.86 3219.50 -23.65 (-0.73%)
var_size_seq_scan q21 2467.49 2478.79 -11.30 (-0.46%)
var_size_seq_scan q22 130.11 130.77 -0.65 (-0.50%)

Copy link

Benchmark Result

Master commit hash: 9eceda0ad3e50053b4410c665753af59228efc63
Branch commit hash: 26ca89e52e651437706fb6c026d4fb2c0a167f77

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 655.06 654.67 0.39 (0.06%)
aggregation q28 12220.88 12206.54 14.34 (0.12%)
filter q14 134.38 135.51 -1.12 (-0.83%)
filter q15 135.00 132.54 2.46 (1.85%)
filter q16 316.25 316.98 -0.73 (-0.23%)
filter q17 455.91 458.18 -2.27 (-0.49%)
filter q18 2007.02 1943.55 63.46 (3.27%)
fixed_size_expr_evaluator q07 547.23 546.74 0.48 (0.09%)
fixed_size_expr_evaluator q08 756.14 757.63 -1.48 (-0.20%)
fixed_size_expr_evaluator q09 756.80 758.37 -1.57 (-0.21%)
fixed_size_expr_evaluator q10 249.10 249.01 0.09 (0.04%)
fixed_size_expr_evaluator q11 245.91 244.72 1.19 (0.49%)
fixed_size_expr_evaluator q12 242.12 242.84 -0.71 (-0.29%)
fixed_size_expr_evaluator q13 1478.32 1484.92 -6.60 (-0.44%)
fixed_size_seq_scan q23 125.37 126.70 -1.34 (-1.05%)
join q31 12.55 9.54 3.00 (31.47%)
ldbc_snb_ic q35 795.33 803.41 -8.07 (-1.00%)
ldbc_snb_ic q36 41.43 50.68 -9.25 (-18.25%)
ldbc_snb_is q32 8.43 8.93 -0.50 (-5.59%)
ldbc_snb_is q33 15.98 18.44 -2.46 (-13.34%)
ldbc_snb_is q34 7.92 8.32 -0.40 (-4.81%)
multi-rel multi-rel-large-scan 2962.73 3550.93 -588.20 (-16.56%)
multi-rel multi-rel-lookup 58.71 42.47 16.24 (38.25%)
multi-rel multi-rel-small-scan 48.25 49.30 -1.06 (-2.14%)
order_by q25 138.42 139.58 -1.16 (-0.83%)
order_by q26 459.91 458.34 1.56 (0.34%)
order_by q27 1427.71 1419.30 8.41 (0.59%)
scan_after_filter q01 182.11 181.30 0.81 (0.45%)
scan_after_filter q02 171.83 169.59 2.23 (1.32%)
shortest_path_ldbc100 q37 3474.78 3396.88 77.91 (2.29%)
shortest_path_ldbc100 q39 52.64 56.75 -4.11 (-7.24%)
shortest_path_ldbc100 q40 71.96 83.07 -11.11 (-13.37%)
var_size_expr_evaluator q03 2077.66 2084.99 -7.33 (-0.35%)
var_size_expr_evaluator q04 2252.80 2255.06 -2.25 (-0.10%)
var_size_expr_evaluator q05 2627.31 2630.77 -3.46 (-0.13%)
var_size_expr_evaluator q06 1392.08 1393.90 -1.82 (-0.13%)
var_size_seq_scan q19 1485.85 1482.05 3.80 (0.26%)
var_size_seq_scan q20 3236.80 3219.50 17.30 (0.54%)
var_size_seq_scan q21 2479.91 2478.79 1.12 (0.05%)
var_size_seq_scan q22 130.77 130.77 0.00 (0.00%)

@benjaminwinger benjaminwinger force-pushed the windows-read-only branch 6 times, most recently from 7d8270b to d12f370 Compare September 13, 2024 19:24
Copy link

Benchmark Result

Master commit hash: a0f987b36e87b36df57272b8c03c81abef266139
Branch commit hash: 3f709bd8b42025f2c05f09944e5430593ece009c

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 655.46 654.84 0.62 (0.09%)
aggregation q28 11896.52 10975.08 921.44 (8.40%)
filter q14 133.57 136.65 -3.08 (-2.25%)
filter q15 135.87 136.91 -1.04 (-0.76%)
filter q16 319.69 319.48 0.21 (0.07%)
filter q17 456.24 457.44 -1.20 (-0.26%)
filter q18 1913.67 1897.03 16.64 (0.88%)
fixed_size_expr_evaluator q07 555.90 546.71 9.19 (1.68%)
fixed_size_expr_evaluator q08 761.78 760.91 0.87 (0.11%)
fixed_size_expr_evaluator q09 772.22 762.40 9.81 (1.29%)
fixed_size_expr_evaluator q10 248.66 248.33 0.33 (0.13%)
fixed_size_expr_evaluator q11 242.94 242.73 0.21 (0.09%)
fixed_size_expr_evaluator q12 243.97 243.79 0.18 (0.07%)
fixed_size_expr_evaluator q13 1496.90 1493.20 3.69 (0.25%)
fixed_size_seq_scan q23 123.48 126.85 -3.37 (-2.66%)
join q31 11.93 12.91 -0.98 (-7.57%)
ldbc_snb_ic q35 465.35 434.10 31.25 (7.20%)
ldbc_snb_ic q36 43.08 38.72 4.36 (11.26%)
ldbc_snb_is q32 9.26 8.82 0.44 (4.97%)
ldbc_snb_is q33 19.11 15.55 3.56 (22.88%)
ldbc_snb_is q34 8.54 7.99 0.55 (6.90%)
multi-rel multi-rel-large-scan 1878.74 1882.75 -4.01 (-0.21%)
multi-rel multi-rel-lookup 69.94 57.01 12.93 (22.69%)
multi-rel multi-rel-small-scan 47.22 59.46 -12.23 (-20.58%)
order_by q25 137.77 138.83 -1.06 (-0.76%)
order_by q26 457.93 455.86 2.07 (0.45%)
order_by q27 1419.63 1413.07 6.56 (0.46%)
scan_after_filter q01 181.08 183.72 -2.64 (-1.44%)
scan_after_filter q02 169.91 170.19 -0.28 (-0.17%)
shortest_path_ldbc100 q37 3493.02 4867.26 -1374.24 (-28.23%)
shortest_path_ldbc100 q39 52.72 58.71 -5.98 (-10.19%)
shortest_path_ldbc100 q40 78.44 74.03 4.41 (5.95%)
var_size_expr_evaluator q03 2069.94 2071.26 -1.32 (-0.06%)
var_size_expr_evaluator q04 2218.57 2212.00 6.58 (0.30%)
var_size_expr_evaluator q05 2567.76 2547.51 20.25 (0.79%)
var_size_expr_evaluator q06 1401.03 1407.29 -6.26 (-0.44%)
var_size_seq_scan q19 1473.90 1478.03 -4.13 (-0.28%)
var_size_seq_scan q20 3190.38 3181.44 8.94 (0.28%)
var_size_seq_scan q21 2450.60 2442.88 7.73 (0.32%)
var_size_seq_scan q22 132.75 131.20 1.54 (1.18%)

@benjaminwinger benjaminwinger merged commit d491c4f into master Sep 16, 2024
25 checks passed
@benjaminwinger benjaminwinger deleted the windows-read-only branch September 16, 2024 12:33
ted-wq-x pushed a commit to ted-wq-x/kuzu that referenced this pull request Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants