Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[enhance](column) add has_enough_capacity function interface for column #47364

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

[enhance](column) add has_enough_capacity function interface for column #47364

wants to merge 8 commits into from

Conversation

zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Jan 23, 2025 •
edited
Loading

What problem does this PR solve?

Problem Summary:

  1. add has_enough_capacity function interface for column
  2. change the sorter conditions for sort data, when need_bytes > capacity_bytes - used_bytes,
    means the left bytes is not enough so need call do_sort().
  3. fix column string overflow in analytic operator, the fixed must in this PR, as the test case without it will failed

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32542 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 84cec8ca64d6fa760cf91e95dddd16d56307bb7e, data reload: false

------ Round 1 ----------------------------------
q1	17589	5532	5370	5370
q2	2058	314	173	173
q3	10515	1287	750	750
q4	10236	1012	548	548
q5	7623	2423	2179	2179
q6	189	168	132	132
q7	929	786	618	618
q8	9250	1407	1227	1227
q9	5152	4974	4959	4959
q10	6839	2378	1892	1892
q11	471	267	254	254
q12	361	374	227	227
q13	17774	3744	3105	3105
q14	242	229	207	207
q15	519	468	456	456
q16	637	641	591	591
q17	589	878	331	331
q18	7204	6524	6397	6397
q19	1935	956	569	569
q20	314	316	199	199
q21	2989	2218	2052	2052
q22	371	347	306	306
Total cold run time: 103786 ms
Total hot run time: 32542 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5597	5495	5514	5495
q2	237	354	233	233
q3	2279	2644	2277	2277
q4	1431	1813	1361	1361
q5	4322	4747	4734	4734
q6	174	163	128	128
q7	2058	1960	1845	1845
q8	2639	2837	2695	2695
q9	7306	7190	7256	7190
q10	2994	3273	2886	2886
q11	579	507	486	486
q12	656	723	570	570
q13	3519	3949	3285	3285
q14	299	297	275	275
q15	503	470	450	450
q16	649	699	653	653
q17	1247	1735	1280	1280
q18	7644	7488	7312	7312
q19	886	1112	1160	1112
q20	1975	2035	1883	1883
q21	5872	5252	4990	4990
q22	668	598	576	576
Total cold run time: 53534 ms
Total hot run time: 51716 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191264 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 84cec8ca64d6fa760cf91e95dddd16d56307bb7e, data reload: false

query1	1309	919	935	919
query2	6187	2046	2031	2031
query3	11131	4651	4683	4651
query4	32527	23211	23371	23211
query5	4390	611	458	458
query6	287	199	189	189
query7	3975	499	310	310
query8	287	252	227	227
query9	9493	2676	2659	2659
query10	467	320	262	262
query11	17656	15116	14934	14934
query12	156	105	105	105
query13	1560	529	406	406
query14	9971	6381	7293	6381
query15	241	213	189	189
query16	8049	687	502	502
query17	1610	770	589	589
query18	2154	444	336	336
query19	218	193	168	168
query20	121	121	122	121
query21	209	125	109	109
query22	4713	4607	4536	4536
query23	34155	33497	33139	33139
query24	6638	2295	2362	2295
query25	496	451	398	398
query26	763	285	162	162
query27	1989	476	334	334
query28	5379	2496	2492	2492
query29	667	558	430	430
query30	218	198	158	158
query31	956	870	829	829
query32	91	62	58	58
query33	539	358	306	306
query34	732	887	514	514
query35	806	832	768	768
query36	1023	1056	964	964
query37	142	104	74	74
query38	4310	4260	4261	4260
query39	1480	1415	1435	1415
query40	206	116	98	98
query41	51	51	47	47
query42	117	102	103	102
query43	546	535	505	505
query44	1322	812	807	807
query45	180	173	170	170
query46	870	1059	647	647
query47	1956	1973	1876	1876
query48	378	402	323	323
query49	732	480	411	411
query50	650	681	416	416
query51	4330	4340	4196	4196
query52	108	102	93	93
query53	231	261	184	184
query54	481	498	403	403
query55	79	77	78	77
query56	253	268	238	238
query57	1242	1205	1121	1121
query58	263	243	242	242
query59	3297	3273	3180	3180
query60	273	267	256	256
query61	120	112	110	110
query62	798	722	697	697
query63	225	193	186	186
query64	3188	1040	661	661
query65	3287	3228	3281	3228
query66	785	393	310	310
query67	16354	15746	15445	15445
query68	5775	819	538	538
query69	487	290	253	253
query70	1203	1154	1150	1150
query71	387	295	246	246
query72	5905	3816	3881	3816
query73	657	741	351	351
query74	10253	9129	8885	8885
query75	3194	3144	2677	2677
query76	3091	1175	760	760
query77	462	352	275	275
query78	10050	9984	9430	9430
query79	2927	796	585	585
query80	701	520	531	520
query81	495	272	242	242
query82	210	152	125	125
query83	170	168	154	154
query84	235	90	73	73
query85	744	345	286	286
query86	386	323	304	304
query87	4360	4433	4280	4280
query88	4795	2169	2152	2152
query89	386	321	290	290
query90	1910	192	186	186
query91	144	137	107	107
query92	67	56	49	49
query93	2639	893	534	534
query94	653	405	286	286
query95	341	261	250	250
query96	503	610	284	284
query97	2810	2889	2807	2807
query98	218	195	200	195
query99	1287	1390	1256	1256
Total cold run time: 286862 ms
Total hot run time: 191264 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.91 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 84cec8ca64d6fa760cf91e95dddd16d56307bb7e, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.64	0.10	0.11
query5	0.40	0.42	0.40
query6	1.14	0.64	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.60	0.49	0.49
query10	0.56	0.57	0.54
query11	0.15	0.10	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.60
query14	2.84	2.90	2.82
query15	0.89	0.82	0.82
query16	0.37	0.38	0.38
query17	1.03	1.04	1.05
query18	0.23	0.20	0.21
query19	1.98	1.78	2.02
query20	0.01	0.01	0.01
query21	15.36	1.01	0.61
query22	0.76	0.73	0.54
query23	15.47	1.38	0.54
query24	3.26	1.69	1.17
query25	0.26	0.15	0.14
query26	0.16	0.14	0.13
query27	0.05	0.06	0.05
query28	13.95	0.96	0.43
query29	12.58	3.99	3.29
query30	0.25	0.09	0.07
query31	2.82	0.58	0.38
query32	3.22	0.55	0.48
query33	2.94	2.98	3.01
query34	16.60	5.21	4.56
query35	4.56	4.51	4.57
query36	0.83	0.49	0.49
query37	0.09	0.06	0.06
query38	0.05	0.03	0.03
query39	0.04	0.02	0.03
query40	0.17	0.13	0.14
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.59 s
Total hot run time: 30.91 s

@zhangstar333 zhangstar333 changed the title [enhance](column) add capacity_bytes function interface for column [enhance](column) add has_enough_capacity function interface for column Jan 24, 2025
@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32491 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 49125258d92b093a4bccd52c18442aae94499a17, data reload: false

------ Round 1 ----------------------------------
q1	17600	5619	5377	5377
q2	2054	312	174	174
q3	10492	1261	754	754
q4	10233	972	523	523
q5	7882	2389	2225	2225
q6	197	168	132	132
q7	905	755	602	602
q8	9230	1346	1149	1149
q9	5221	4929	4939	4929
q10	6863	2332	1895	1895
q11	456	273	250	250
q12	341	350	217	217
q13	17766	3740	3164	3164
q14	242	231	206	206
q15	531	474	461	461
q16	629	625	595	595
q17	564	874	326	326
q18	7189	6549	6517	6517
q19	1831	960	560	560
q20	301	322	181	181
q21	2761	2166	1942	1942
q22	371	338	312	312
Total cold run time: 103659 ms
Total hot run time: 32491 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5516	5487	5503	5487
q2	234	327	231	231
q3	2296	2648	2366	2366
q4	1431	1807	1347	1347
q5	4349	4723	4648	4648
q6	166	159	130	130
q7	2053	1948	1805	1805
q8	2618	2848	2718	2718
q9	7399	7225	7263	7225
q10	3064	3306	2822	2822
q11	593	521	489	489
q12	634	734	584	584
q13	3474	3998	3321	3321
q14	282	315	276	276
q15	516	482	473	473
q16	657	682	665	665
q17	1246	1753	1274	1274
q18	7732	7640	7278	7278
q19	803	1167	1121	1121
q20	1993	2079	1900	1900
q21	5814	5287	5043	5043
q22	595	579	554	554
Total cold run time: 53465 ms
Total hot run time: 51757 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192171 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 49125258d92b093a4bccd52c18442aae94499a17, data reload: false

query1	1303	965	923	923
query2	6319	2090	2009	2009
query3	11125	4606	4644	4606
query4	32574	23267	23153	23153
query5	3679	607	463	463
query6	287	208	190	190
query7	3979	489	317	317
query8	304	253	230	230
query9	9049	2630	2612	2612
query10	449	321	261	261
query11	17852	15456	14901	14901
query12	177	106	102	102
query13	1577	546	392	392
query14	9617	7289	7415	7289
query15	222	212	200	200
query16	7909	659	472	472
query17	1566	776	566	566
query18	2148	404	318	318
query19	215	182	166	166
query20	131	130	113	113
query21	202	126	112	112
query22	4596	4665	4503	4503
query23	34506	33545	33415	33415
query24	6525	2280	2268	2268
query25	508	481	422	422
query26	837	329	164	164
query27	1996	475	332	332
query28	5477	2530	2469	2469
query29	671	569	428	428
query30	215	189	162	162
query31	962	890	829	829
query32	85	57	63	57
query33	471	359	312	312
query34	778	865	520	520
query35	794	833	779	779
query36	1025	1062	961	961
query37	127	106	90	90
query38	4230	4362	4391	4362
query39	1492	1481	1475	1475
query40	217	118	105	105
query41	56	55	56	55
query42	129	112	105	105
query43	526	528	497	497
query44	1445	840	833	833
query45	189	184	173	173
query46	892	1077	663	663
query47	1913	1917	1849	1849
query48	386	430	335	335
query49	754	524	451	451
query50	636	678	396	396
query51	4270	4290	4179	4179
query52	107	111	93	93
query53	228	255	190	190
query54	512	555	422	422
query55	91	80	80	80
query56	274	271	245	245
query57	1234	1223	1170	1170
query58	269	229	223	223
query59	3148	3224	3038	3038
query60	305	290	297	290
query61	114	118	131	118
query62	787	728	665	665
query63	231	198	197	197
query64	3640	1024	676	676
query65	3307	3250	3224	3224
query66	781	425	299	299
query67	16271	16030	15437	15437
query68	5811	819	520	520
query69	485	297	263	263
query70	1184	1179	1130	1130
query71	376	282	254	254
query72	6208	3828	3841	3828
query73	652	748	366	366
query74	10464	8971	8933	8933
query75	3186	3156	2675	2675
query76	3097	1202	753	753
query77	458	360	280	280
query78	10146	10070	9306	9306
query79	3061	811	616	616
query80	667	507	437	437
query81	508	270	241	241
query82	215	144	126	126
query83	174	175	214	175
query84	235	97	68	68
query85	726	338	303	303
query86	386	292	274	274
query87	4367	4452	4332	4332
query88	4879	2204	2169	2169
query89	407	320	287	287
query90	1870	186	188	186
query91	132	143	106	106
query92	70	53	54	53
query93	2532	858	532	532
query94	657	407	294	294
query95	331	265	253	253
query96	487	610	280	280
query97	2762	2822	2722	2722
query98	232	212	198	198
query99	1294	1343	1260	1260
Total cold run time: 286750 ms
Total hot run time: 192171 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.98 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 49125258d92b093a4bccd52c18442aae94499a17, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.11
query5	0.40	0.42	0.41
query6	1.12	0.64	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.50	0.49
query10	0.55	0.56	0.54
query11	0.15	0.11	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.59
query14	2.82	2.82	2.75
query15	0.90	0.82	0.84
query16	0.36	0.38	0.39
query17	1.06	1.06	1.02
query18	0.23	0.21	0.21
query19	1.96	1.83	2.01
query20	0.02	0.01	0.01
query21	15.36	0.97	0.56
query22	0.75	0.86	0.68
query23	15.16	1.45	0.56
query24	3.27	1.17	1.65
query25	0.18	0.13	0.12
query26	0.29	0.15	0.14
query27	0.05	0.06	0.05
query28	14.43	0.96	0.43
query29	12.58	3.94	3.28
query30	0.25	0.09	0.07
query31	2.84	0.62	0.37
query32	3.22	0.54	0.45
query33	2.99	2.98	3.04
query34	16.58	5.27	4.59
query35	4.59	4.59	4.56
query36	0.66	0.49	0.47
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.12	0.13
query41	0.08	0.02	0.02
query42	0.04	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.67 s
Total hot run time: 30.98 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 41.96% (10949/26096)
Line Coverage: 32.24% (92478/286820)
Region Coverage: 31.37% (47389/151053)
Branch Coverage: 27.43% (24008/87512)
Coverage Report: http://coverage.selectdb-in.cc/coverage/49125258d92b093a4bccd52c18442aae94499a17_49125258d92b093a4bccd52c18442aae94499a17/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32439 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8fec35458b22867d35205aea5e96dbdd694c114b, data reload: false

------ Round 1 ----------------------------------
q1	17591	5467	5420	5420
q2	2056	313	180	180
q3	10473	1221	734	734
q4	10221	994	535	535
q5	7720	2410	2145	2145
q6	205	171	135	135
q7	905	785	603	603
q8	9233	1364	1161	1161
q9	5237	4969	4949	4949
q10	6842	2350	1871	1871
q11	470	270	250	250
q12	336	353	216	216
q13	17786	3729	3141	3141
q14	230	227	205	205
q15	527	478	480	478
q16	639	616	584	584
q17	564	885	340	340
q18	7062	6597	6468	6468
q19	1634	967	535	535
q20	309	330	196	196
q21	2814	2200	1980	1980
q22	368	342	313	313
Total cold run time: 103222 ms
Total hot run time: 32439 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5576	5534	5531	5531
q2	242	336	239	239
q3	2303	2669	2341	2341
q4	1470	1845	1360	1360
q5	4324	4765	4653	4653
q6	165	165	136	136
q7	2076	1966	1870	1870
q8	2636	2848	2751	2751
q9	7352	7198	7238	7198
q10	3081	3318	2814	2814
q11	589	504	493	493
q12	652	734	570	570
q13	3438	3967	3287	3287
q14	277	301	286	286
q15	506	479	476	476
q16	631	693	643	643
q17	1225	1746	1255	1255
q18	7693	7496	7538	7496
q19	790	1104	1112	1104
q20	2024	2035	1890	1890
q21	5783	5356	4976	4976
q22	617	596	554	554
Total cold run time: 53450 ms
Total hot run time: 51923 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191992 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8fec35458b22867d35205aea5e96dbdd694c114b, data reload: false

query1	1306	946	944	944
query2	6227	2136	2015	2015
query3	11104	4693	4327	4327
query4	61504	34728	23150	23150
query5	5358	615	466	466
query6	328	180	169	169
query7	4978	509	309	309
query8	298	232	229	229
query9	6182	2700	2665	2665
query10	412	323	260	260
query11	15632	14902	15486	14902
query12	164	110	108	108
query13	1042	552	441	441
query14	10779	7420	7676	7420
query15	210	201	198	198
query16	7043	629	486	486
query17	1101	711	569	569
query18	1581	410	326	326
query19	197	185	194	185
query20	114	108	110	108
query21	221	126	97	97
query22	4846	4740	4635	4635
query23	33942	33255	33509	33255
query24	5542	2294	2282	2282
query25	462	454	395	395
query26	629	281	153	153
query27	1607	501	341	341
query28	4070	2509	2487	2487
query29	540	569	449	449
query30	213	203	158	158
query31	941	865	807	807
query32	72	58	61	58
query33	403	350	298	298
query34	724	881	525	525
query35	801	850	775	775
query36	1064	1032	937	937
query37	125	103	78	78
query38	4316	4246	4300	4246
query39	1485	1450	1435	1435
query40	208	116	101	101
query41	60	49	52	49
query42	117	107	106	106
query43	524	531	502	502
query44	1343	815	826	815
query45	189	172	173	172
query46	882	1069	676	676
query47	1933	1948	1855	1855
query48	376	412	325	325
query49	700	498	412	412
query50	671	685	409	409
query51	4253	4282	4214	4214
query52	108	103	101	101
query53	231	256	204	204
query54	493	506	429	429
query55	82	81	80	80
query56	254	270	262	262
query57	1231	1243	1167	1167
query58	255	261	253	253
query59	3243	3201	3202	3201
query60	306	291	275	275
query61	140	135	145	135
query62	748	729	667	667
query63	227	198	192	192
query64	1237	1103	757	757
query65	3247	3137	3174	3137
query66	687	399	292	292
query67	15998	15659	15532	15532
query68	5076	824	526	526
query69	483	292	259	259
query70	1163	1099	1115	1099
query71	401	277	266	266
query72	6186	3837	3837	3837
query73	826	756	363	363
query74	9939	9163	8567	8567
query75	3220	3149	2681	2681
query76	3770	1161	758	758
query77	476	375	269	269
query78	10273	10044	9404	9404
query79	3386	796	579	579
query80	1609	531	441	441
query81	572	270	236	236
query82	358	155	132	132
query83	269	168	157	157
query84	286	94	74	74
query85	789	343	313	313
query86	455	309	302	302
query87	4456	4424	4455	4424
query88	4512	2220	2180	2180
query89	393	330	285	285
query90	1595	195	193	193
query91	134	134	109	109
query92	67	56	53	53
query93	2429	874	534	534
query94	753	412	275	275
query95	339	266	257	257
query96	491	608	285	285
query97	2840	2851	2701	2701
query98	226	194	225	194
query99	1264	1388	1231	1231
Total cold run time: 306342 ms
Total hot run time: 191992 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 8fec35458b22867d35205aea5e96dbdd694c114b, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.03
query3	0.25	0.06	0.07
query4	1.62	0.10	0.10
query5	0.39	0.43	0.39
query6	1.13	0.66	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.58	0.49	0.50
query10	0.55	0.56	0.56
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.62	0.61	0.59
query14	2.76	2.75	2.73
query15	0.90	0.82	0.82
query16	0.38	0.40	0.38
query17	1.07	1.07	1.06
query18	0.22	0.21	0.21
query19	1.96	1.79	2.00
query20	0.01	0.02	0.01
query21	15.38	0.97	0.59
query22	0.74	0.89	0.70
query23	15.25	1.42	0.61
query24	2.93	1.83	1.44
query25	0.19	0.10	0.13
query26	0.34	0.16	0.13
query27	0.05	0.05	0.06
query28	13.92	0.96	0.43
query29	12.58	4.01	3.28
query30	0.26	0.09	0.06
query31	2.82	0.61	0.38
query32	3.22	0.55	0.45
query33	3.01	3.01	2.98
query34	16.67	5.16	4.55
query35	4.54	4.61	4.56
query36	0.65	0.50	0.48
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.03	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 105.94 s
Total hot run time: 31.25 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 42.04% (10971/26098)
Line Coverage: 32.33% (92729/286861)
Region Coverage: 31.48% (47552/151064)
Branch Coverage: 27.51% (24075/87518)
Coverage Report: http://coverage.selectdb-in.cc/coverage/8fec35458b22867d35205aea5e96dbdd694c114b_8fec35458b22867d35205aea5e96dbdd694c114b/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32380 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 52dc039fc49950b8534e18b3ebc7cb7b51c96a23, data reload: false

------ Round 1 ----------------------------------
q1	17601	5599	5353	5353
q2	2048	313	165	165
q3	10407	1252	735	735
q4	10225	989	537	537
q5	7549	2384	2193	2193
q6	186	181	132	132
q7	907	769	598	598
q8	9236	1361	1157	1157
q9	5240	4953	4917	4917
q10	6842	2326	1889	1889
q11	472	276	251	251
q12	352	358	226	226
q13	17774	3767	3171	3171
q14	225	220	213	213
q15	516	473	473	473
q16	625	612	583	583
q17	555	886	321	321
q18	7275	6563	6445	6445
q19	1558	955	554	554
q20	316	324	190	190
q21	2844	2174	1974	1974
q22	373	340	303	303
Total cold run time: 103126 ms
Total hot run time: 32380 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5504	5484	5515	5484
q2	250	330	231	231
q3	2242	2701	2314	2314
q4	1438	1827	1420	1420
q5	4353	4714	4685	4685
q6	172	160	129	129
q7	2047	1963	1834	1834
q8	2641	2779	2783	2779
q9	7419	7372	7344	7344
q10	3093	3306	2798	2798
q11	577	509	489	489
q12	640	753	561	561
q13	3470	3916	3242	3242
q14	280	308	281	281
q15	518	502	459	459
q16	666	653	646	646
q17	1218	1775	1273	1273
q18	7746	7569	7479	7479
q19	805	1228	1082	1082
q20	2020	2047	1906	1906
q21	5770	5204	4910	4910
q22	625	651	593	593
Total cold run time: 53494 ms
Total hot run time: 51939 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191187 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 52dc039fc49950b8534e18b3ebc7cb7b51c96a23, data reload: false

query1	1339	969	942	942
query2	6211	1997	2039	1997
query3	11140	4656	4494	4494
query4	61707	29285	22961	22961
query5	5520	597	468	468
query6	447	201	184	184
query7	5530	518	296	296
query8	323	226	216	216
query9	8175	2708	2710	2708
query10	440	301	255	255
query11	17929	15018	15432	15018
query12	150	109	110	109
query13	1415	561	433	433
query14	10712	6552	7475	6552
query15	213	203	190	190
query16	7234	684	446	446
query17	1117	758	607	607
query18	1949	418	319	319
query19	211	184	158	158
query20	119	118	113	113
query21	216	123	104	104
query22	4400	4725	4529	4529
query23	34189	33436	33981	33436
query24	5568	2237	2274	2237
query25	474	470	388	388
query26	643	280	156	156
query27	1605	466	334	334
query28	4078	2526	2487	2487
query29	532	550	416	416
query30	203	191	159	159
query31	938	867	829	829
query32	69	59	58	58
query33	441	367	324	324
query34	740	851	509	509
query35	865	847	767	767
query36	1031	1049	967	967
query37	126	100	77	77
query38	4302	4323	4120	4120
query39	1504	1465	1418	1418
query40	206	115	102	102
query41	53	49	48	48
query42	117	104	101	101
query43	524	523	489	489
query44	1357	830	864	830
query45	187	178	171	171
query46	876	1056	681	681
query47	1971	1912	1884	1884
query48	405	417	328	328
query49	705	510	406	406
query50	660	675	412	412
query51	4351	4367	4253	4253
query52	111	108	102	102
query53	237	259	180	180
query54	477	496	434	434
query55	92	87	84	84
query56	252	291	260	260
query57	1202	1195	1127	1127
query58	246	244	227	227
query59	3248	3318	2957	2957
query60	321	278	269	269
query61	120	113	109	109
query62	761	718	676	676
query63	233	193	205	193
query64	1268	1027	664	664
query65	3255	3314	3118	3118
query66	758	393	301	301
query67	16217	15748	15557	15557
query68	4124	834	550	550
query69	471	303	318	303
query70	1180	1143	1130	1130
query71	392	286	252	252
query72	6311	3960	3837	3837
query73	701	752	369	369
query74	9278	9151	8808	8808
query75	3174	3145	2723	2723
query76	3239	1185	764	764
query77	492	355	266	266
query78	10317	10071	9445	9445
query79	3438	793	600	600
query80	862	512	452	452
query81	505	277	238	238
query82	941	153	118	118
query83	180	167	148	148
query84	289	96	73	73
query85	771	346	304	304
query86	426	314	308	308
query87	4457	4444	4300	4300
query88	4408	2197	2194	2194
query89	399	322	292	292
query90	1552	184	188	184
query91	134	146	108	108
query92	70	56	52	52
query93	2630	821	536	536
query94	859	378	292	292
query95	321	262	254	254
query96	507	615	287	287
query97	2783	2845	2796	2796
query98	226	201	202	201
query99	1351	1379	1259	1259
Total cold run time: 310803 ms
Total hot run time: 191187 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.6 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 52dc039fc49950b8534e18b3ebc7cb7b51c96a23, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.42	0.45	0.41
query6	1.15	0.65	0.67
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.61	0.50	0.50
query10	0.56	0.56	0.54
query11	0.15	0.10	0.11
query12	0.15	0.12	0.11
query13	0.61	0.58	0.60
query14	2.82	2.88	2.83
query15	0.90	0.83	0.83
query16	0.37	0.38	0.39
query17	1.06	0.99	0.98
query18	0.23	0.22	0.20
query19	1.84	1.76	2.07
query20	0.02	0.01	0.02
query21	15.36	0.92	0.59
query22	0.76	0.79	0.87
query23	15.10	1.45	0.53
query24	3.71	0.77	1.53
query25	0.20	0.11	0.10
query26	0.31	0.14	0.13
query27	0.07	0.05	0.05
query28	14.07	0.98	0.43
query29	12.55	4.04	3.26
query30	0.25	0.09	0.07
query31	2.83	0.63	0.38
query32	3.23	0.57	0.45
query33	2.97	3.02	3.15
query34	16.60	5.28	4.56
query35	4.59	4.54	4.61
query36	0.66	0.50	0.48
query37	0.10	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.02	0.02
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.68 s
Total hot run time: 30.6 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 42.03% (10970/26098)
Line Coverage: 32.33% (92732/286858)
Region Coverage: 31.47% (47542/151062)
Branch Coverage: 27.51% (24078/87516)
Coverage Report: http://coverage.selectdb-in.cc/coverage/52dc039fc49950b8534e18b3ebc7cb7b51c96a23_52dc039fc49950b8534e18b3ebc7cb7b51c96a23/report/index.html

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32486 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5d40deedd1022e7842b1e1a6191fde238372826, data reload: false

------ Round 1 ----------------------------------
q1	17612	5422	5397	5397
q2	2048	297	166	166
q3	10476	1223	741	741
q4	10241	966	526	526
q5	8361	2424	2191	2191
q6	208	168	131	131
q7	925	762	587	587
q8	9236	1376	1239	1239
q9	5356	4958	4989	4958
q10	6846	2342	1885	1885
q11	462	279	246	246
q12	349	362	215	215
q13	17768	3641	3085	3085
q14	235	224	220	220
q15	521	463	463	463
q16	629	601	590	590
q17	550	879	314	314
q18	7048	6573	6576	6573
q19	1855	958	529	529
q20	297	315	186	186
q21	2851	2167	1941	1941
q22	364	330	303	303
Total cold run time: 104238 ms
Total hot run time: 32486 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5557	5475	5452	5452
q2	245	328	239	239
q3	2259	2679	2352	2352
q4	1417	1818	1366	1366
q5	4281	4746	4679	4679
q6	168	163	128	128
q7	2072	2017	1848	1848
q8	2630	2812	2719	2719
q9	7384	7372	7342	7342
q10	3006	3323	2711	2711
q11	561	502	480	480
q12	714	769	627	627
q13	3490	3855	3233	3233
q14	287	306	291	291
q15	504	472	466	466
q16	651	680	652	652
q17	1233	1759	1259	1259
q18	7747	7493	7382	7382
q19	793	928	1117	928
q20	2008	2065	1878	1878
q21	5884	5184	5096	5096
q22	594	613	546	546
Total cold run time: 53485 ms
Total hot run time: 51674 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192172 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5d40deedd1022e7842b1e1a6191fde238372826, data reload: false

query1	1339	974	934	934
query2	6261	2001	1985	1985
query3	11056	4585	4670	4585
query4	61075	28698	23001	23001
query5	5476	577	455	455
query6	442	190	182	182
query7	5540	494	303	303
query8	337	236	232	232
query9	8200	2710	2714	2710
query10	442	317	262	262
query11	17403	15507	15448	15448
query12	166	115	109	109
query13	1469	546	439	439
query14	10617	7831	7347	7347
query15	200	201	185	185
query16	7104	659	484	484
query17	1143	766	603	603
query18	1300	420	321	321
query19	207	199	162	162
query20	117	115	118	115
query21	218	123	105	105
query22	4633	4748	4473	4473
query23	33975	33369	34215	33369
query24	5635	2289	2374	2289
query25	454	446	406	406
query26	663	252	151	151
query27	1782	479	344	344
query28	4164	2516	2487	2487
query29	564	534	425	425
query30	223	188	157	157
query31	910	899	838	838
query32	68	59	57	57
query33	441	365	293	293
query34	761	871	527	527
query35	807	822	768	768
query36	1026	1039	971	971
query37	123	91	97	91
query38	4296	4315	4178	4178
query39	1480	1627	1439	1439
query40	203	121	100	100
query41	49	48	48	48
query42	115	102	102	102
query43	515	527	495	495
query44	1319	843	832	832
query45	186	193	168	168
query46	870	1074	666	666
query47	1892	1930	1833	1833
query48	401	422	329	329
query49	734	484	398	398
query50	633	685	406	406
query51	4241	4307	4259	4259
query52	104	98	97	97
query53	224	252	192	192
query54	508	519	409	409
query55	88	83	82	82
query56	241	267	257	257
query57	1205	1223	1166	1166
query58	244	232	234	232
query59	3220	3299	2883	2883
query60	279	269	245	245
query61	112	110	115	110
query62	713	720	643	643
query63	221	185	198	185
query64	1414	1017	649	649
query65	3297	3137	3152	3137
query66	693	440	292	292
query67	15872	15539	15456	15456
query68	4428	823	539	539
query69	489	288	254	254
query70	1163	1117	1130	1117
query71	422	286	254	254
query72	6139	3831	3830	3830
query73	746	762	365	365
query74	10148	8854	8880	8854
query75	3225	3160	2697	2697
query76	3610	1161	765	765
query77	472	401	279	279
query78	10034	9940	9357	9357
query79	3348	836	597	597
query80	1707	525	435	435
query81	565	269	232	232
query82	351	148	117	117
query83	262	168	153	153
query84	295	97	76	76
query85	766	355	293	293
query86	464	311	305	305
query87	4406	4461	4330	4330
query88	4395	2213	2192	2192
query89	393	333	291	291
query90	1569	191	194	191
query91	132	134	106	106
query92	65	56	53	53
query93	2651	887	539	539
query94	744	394	311	311
query95	331	261	256	256
query96	520	616	287	287
query97	2772	2890	2785	2785
query98	222	198	191	191
query99	1269	1395	1220	1220
Total cold run time: 309813 ms
Total hot run time: 192172 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.72 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e5d40deedd1022e7842b1e1a6191fde238372826, data reload: false

query1	0.03	0.03	0.04
query2	0.08	0.04	0.04
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.39	0.43	0.41
query6	1.17	0.66	0.66
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.56	0.52	0.50
query10	0.56	0.57	0.54
query11	0.15	0.10	0.10
query12	0.13	0.11	0.11
query13	0.61	0.60	0.60
query14	2.75	2.72	2.85
query15	0.91	0.82	0.83
query16	0.38	0.40	0.38
query17	1.06	0.99	0.98
query18	0.23	0.21	0.20
query19	1.94	1.83	1.94
query20	0.01	0.01	0.01
query21	15.37	0.95	0.58
query22	0.75	0.77	0.56
query23	15.45	1.39	0.61
query24	2.70	0.99	1.17
query25	0.14	0.09	0.15
query26	0.35	0.16	0.12
query27	0.05	0.06	0.06
query28	13.87	1.05	0.43
query29	12.56	4.03	3.34
query30	0.25	0.09	0.07
query31	2.82	0.59	0.40
query32	3.22	0.55	0.46
query33	3.03	3.05	3.02
query34	16.70	5.14	4.55
query35	4.58	4.52	4.56
query36	0.66	0.48	0.47
query37	0.10	0.06	0.06
query38	0.04	0.03	0.04
query39	0.04	0.02	0.03
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.89 s
Total hot run time: 30.72 s

@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32229 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 73a920931ef79d66ab759751904daffe6a121817, data reload: false

------ Round 1 ----------------------------------
q1	17571	5528	5388	5388
q2	2040	323	169	169
q3	10518	1323	734	734
q4	10265	953	528	528
q5	8268	2375	2146	2146
q6	196	164	132	132
q7	892	770	614	614
q8	9224	1312	1169	1169
q9	5341	5018	4874	4874
q10	6854	2338	1875	1875
q11	459	267	254	254
q12	337	356	212	212
q13	17773	3744	3057	3057
q14	218	231	206	206
q15	521	481	475	475
q16	637	605	587	587
q17	563	859	337	337
q18	7252	6510	6466	6466
q19	1416	949	522	522
q20	314	318	191	191
q21	2793	2217	1981	1981
q22	352	329	312	312
Total cold run time: 103804 ms
Total hot run time: 32229 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5529	5478	5511	5478
q2	240	330	237	237
q3	2336	2638	2357	2357
q4	1427	1808	1374	1374
q5	4299	4740	4677	4677
q6	162	157	127	127
q7	2081	1953	1840	1840
q8	2679	2790	2736	2736
q9	7333	7301	7242	7242
q10	2979	3293	2791	2791
q11	576	514	498	498
q12	673	797	637	637
q13	3463	3832	3262	3262
q14	275	307	274	274
q15	511	479	457	457
q16	626	690	636	636
q17	1231	1743	1269	1269
q18	7647	7472	7260	7260
q19	758	997	1113	997
q20	2029	2061	1891	1891
q21	5752	5241	5013	5013
q22	606	571	541	541
Total cold run time: 53212 ms
Total hot run time: 51594 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184269 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 73a920931ef79d66ab759751904daffe6a121817, data reload: false

query1	982	391	360	360
query2	6510	2008	2020	2008
query3	6789	219	219	219
query4	33595	23291	22959	22959
query5	4268	590	461	461
query6	278	192	165	165
query7	4603	519	311	311
query8	319	235	219	219
query9	9314	2712	2695	2695
query10	453	309	246	246
query11	18219	15146	14953	14953
query12	161	112	103	103
query13	1651	532	409	409
query14	9747	6348	7001	6348
query15	240	192	183	183
query16	8097	630	471	471
query17	1610	745	575	575
query18	2126	400	290	290
query19	206	177	159	159
query20	117	117	111	111
query21	213	124	105	105
query22	4253	4296	4356	4296
query23	34261	32935	33093	32935
query24	6594	2231	2313	2231
query25	546	442	375	375
query26	1194	285	155	155
query27	2020	467	329	329
query28	5130	2485	2436	2436
query29	742	535	406	406
query30	231	221	168	168
query31	945	865	820	820
query32	83	64	58	58
query33	523	336	296	296
query34	749	855	495	495
query35	790	805	741	741
query36	955	1007	950	950
query37	115	100	85	85
query38	4140	4152	4020	4020
query39	1486	1381	1389	1381
query40	208	107	97	97
query41	53	50	55	50
query42	122	100	101	100
query43	493	517	472	472
query44	1279	822	833	822
query45	191	171	165	165
query46	846	1037	634	634
query47	1838	1880	1766	1766
query48	373	398	336	336
query49	791	485	399	399
query50	605	669	396	396
query51	4167	4205	4168	4168
query52	105	101	93	93
query53	227	258	198	198
query54	480	510	399	399
query55	84	81	79	79
query56	254	254	242	242
query57	1184	1158	1065	1065
query58	243	230	231	230
query59	2920	3124	2823	2823
query60	291	272	260	260
query61	135	109	115	109
query62	779	705	647	647
query63	217	189	185	185
query64	4646	994	638	638
query65	3199	3158	3197	3158
query66	1063	401	312	312
query67	15809	15606	15591	15591
query68	4599	833	539	539
query69	461	292	256	256
query70	1224	1106	1088	1088
query71	378	285	252	252
query72	5818	3755	3810	3755
query73	653	746	362	362
query74	9122	9150	9064	9064
query75	3151	3170	2690	2690
query76	3152	1195	758	758
query77	476	371	349	349
query78	9892	9936	9277	9277
query79	2860	805	602	602
query80	650	522	442	442
query81	498	278	242	242
query82	278	159	120	120
query83	174	174	160	160
query84	234	90	75	75
query85	775	336	289	289
query86	401	321	300	300
query87	4423	4524	4284	4284
query88	4865	2181	2186	2181
query89	382	331	296	296
query90	1795	188	184	184
query91	133	133	107	107
query92	65	55	54	54
query93	2333	881	536	536
query94	655	398	296	296
query95	335	259	253	253
query96	492	603	287	287
query97	2726	2852	2739	2739
query98	224	219	198	198
query99	1291	1381	1273	1273
Total cold run time: 281372 ms
Total hot run time: 184269 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 73a920931ef79d66ab759751904daffe6a121817, data reload: false

query1	0.04	0.03	0.05
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.60	0.10	0.10
query5	0.41	0.41	0.41
query6	1.17	0.66	0.65
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.59	0.49	0.51
query10	0.55	0.56	0.55
query11	0.14	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.60	0.60
query14	2.72	2.78	2.77
query15	0.89	0.84	0.82
query16	0.36	0.38	0.38
query17	1.03	1.05	1.02
query18	0.23	0.21	0.20
query19	1.86	1.83	1.98
query20	0.01	0.02	0.02
query21	15.38	0.99	0.60
query22	0.74	0.76	0.68
query23	15.32	1.32	0.54
query24	3.53	1.29	1.56
query25	0.27	0.16	0.14
query26	0.33	0.15	0.14
query27	0.08	0.07	0.07
query28	13.75	0.98	0.43
query29	12.60	3.96	3.33
query30	0.26	0.09	0.06
query31	2.82	0.60	0.40
query32	3.23	0.54	0.46
query33	2.96	2.94	3.03
query34	16.80	5.10	4.53
query35	4.51	4.49	4.52
query36	0.64	0.50	0.48
query37	0.10	0.07	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.02	0.03
Total cold run time: 106.44 s
Total hot run time: 31.1 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 42.04% (10971/26098)
Line Coverage: 32.32% (92707/286861)
Region Coverage: 31.47% (47545/151070)
Branch Coverage: 27.51% (24077/87516)
Coverage Report: http://coverage.selectdb-in.cc/coverage/73a920931ef79d66ab759751904daffe6a121817_73a920931ef79d66ab759751904daffe6a121817/report/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HappenLee HappenLee HappenLee left review comments

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zhangstar333 @hello-stephen @doris-robot @BiteTheDDDDt @HappenLee