[fix](planner) fix core when select and filter by slot in old planner #46541

liutang123 · 2025-01-07T08:33:30Z

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
In 2.1.7, if a sql parse failed in nereids planner, it will fallback to old planner.
and old planner maybe create SelectNode with some conjuncts that are not boolean type.

Reproduce:
Create table sql see #46498
query sql1:

select b.c_id   from 
dbgr as b  
left join  
(select c.c_id from lo  where event_date between 20220500 and 20220600 limit 100 )c    
on c.c_id LIMIT  200;

query sql2:

select b.c_id  
from dbgr as b  
left join   
(select c.c_id from lo )c  
on c.c_id  
LIMIT 0, 200;

Because select c.c_id, these sqls will fallback to old planner.
Because on c.c_id is not boolean type, and be will core.
A part of query plan is as follows:

|   1:VOlapScanNode                                                           |
|      TABLE: test.lo(lo), PREAGGREGATION: ON                                 |
|      PREDICATES: (`c`.`c_id` AND (`test`.`lo`.`__DORIS_DELETE_SIGN__` = 0)) |
|      partitions=1/3 (p_202206)                                              |
|      tablets=2/2, tabletList=89678,89680                                    |
|      cardinality=46, avgRowSize=165.54349, numNodes=1                       |
|      pushAggOp=NONE                                                         |
+-----------------------------------------------------------------------------+

A fatal log is as follows:

F20241219 23:13:23.457937 33282 assert_cast.h:58] Bad cast from type:doris::vectorized::ColumnVector<int> to doris::vectorized::ColumnVector<unsigned 
char>
*** Check failure stack trace: ***
    @     0x55bfa043b956  google::LogMessageFatal::~LogMessageFatal()
    @     0x55bf6f3bc070  assert_cast<>()
    @     0x55bf8978d767  doris::vectorized::VExprContext::execute_conjuncts()
    @     0x55bf8978c463  doris::vectorized::VExprContext::execute_conjuncts_and_filter_block()
    @     0x55bf8978bf72  doris::vectorized::VExprContext::filter_block()
    @     0x55bfa035b8e4  doris::pipeline::SelectOperatorX::pull()
    @     0x55bf9fee2b62  doris::pipeline::StreamingOperatorX<>::get_block()
    @     0x55bf9feab54b  doris::pipeline::OperatorXBase::get_block_after_projects()
    @     0x55bfa03dd07c  doris::pipeline::PipelineXTask::execute()
    @     0x55bfa0413e85  doris::pipeline::TaskScheduler::_do_work()
    @     0x55bfa0417dcb  doris::pipeline::TaskScheduler::start()::$_0::operator()()
    @     0x55bfa0417d55  std::__invoke_impl<>()
    @     0x55bfa0417d05  _ZSt10__invoke_rIvRZN5doris8pipeline13TaskScheduler5startEvE3$_0JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES6_E4typeEO
S7_DpOS8_
    @     0x55bfa0417bcd  std::_Function_handler<>::_M_invoke()
    @     0x55bf6e6c9b63  std::function<>::operator()()
    @     0x55bf7289e209  doris::FunctionRunnable::run()
    @     0x55bf728899c0  doris::ThreadPool::dispatch_thread()
    @     0x55bf728b0c24  std::__invoke_impl<>()
    @     0x55bf728b0afd  std::__invoke<>()
    @     0x55bf728b0a85  _ZNSt5_BindIFMN5doris10ThreadPoolEFvvEPS1_EE6__callIvJEJLm0EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE
    @     0x55bf728b092e  std::_Bind<>::operator()<>()
    @     0x55bf728b0845  std::__invoke_impl<>()
    @     0x55bf728b07e5  _ZSt10__invoke_rIvRSt5_BindIFMN5doris10ThreadPoolEFvvEPS2_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESA_E4typeEOSB_D
pOSC_
    @     0x55bf728b048d  std::_Function_handler<>::_M_invoke()
    @     0x55bf6e6c9b63  std::function<>::operator()()
    @     0x55bf728521fc  doris::Thread::supervise_thread()
    @     0x7f4260614ea5  start_thread
    @     0x7f42610439fd  __clone
    @              (nil)  (unknown)

And another:

F20250108 13:07:05.275424 184257 assert_cast.h:58] Bad cast from type:doris::vectorized::ColumnVector<int> to doris::vectorized::ColumnVector<unsigned char>
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/tc/be/src/common/signal_handler.h:421
 1# 0x00007FB73FB31400 in /lib64/libc.so.6
 2# __GI_raise in /lib64/libc.so.6
 3# abort in /lib64/libc.so.6
 4# 0x000055CDAAF0090D in /usr/local/service/doris/lib/be/doris_be
 5# google::LogMessage::SendToLog() in /usr/local/service/doris/lib/be/doris_be
 6# google::LogMessage::Flush() in /usr/local/service/doris/lib/be/doris_be
 7# google::LogMessageFatal::~LogMessageFatal() in /usr/local/service/doris/lib/be/doris_be
 8# doris::vectorized::ColumnVector<unsigned char> const& assert_cast<doris::vectorized::ColumnVector<unsigned char> const&, doris::vec
torized::IColumn const&>(doris::vectorized::IColumn const&) in /usr/local/service/doris/lib/be/doris_be
 9# doris::vectorized::VExprContext::execute_conjuncts(std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std
::shared_ptr<doris::vectorized::VExprContext> > > const&, std::vector<doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<fals
e, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*, std::allocator<doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<fal
se, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*> > const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray<unsigne
d char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*, bool*) at /root/tc/be/src/vec/exprs/vexpr_context
.cpp:181
10# doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector<std::shared_ptr<doris::vectorized::VExprContext>, s
td::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const&, doris::vectorized::Block*, std::vector<unsigned int, std::al
locator<unsigned int> >&, int, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator
>, 16ul, 16ul>&) at /root/tc/be/src/vec/exprs/vexpr_context.cpp:324
11# doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) at /root/tc/b
e/src/olap/rowset/segment_v2/segment_iterator.cpp:2274
12# doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2178
13# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1914
14# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1913
15# doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) in /usr/local/service/doris/lib/be/doris_be
16# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/beta_rowset_reader.cpp:348
17# doris::vectorized::VCollectIterator::Level0Iterator::_refresh() in /usr/local/service/doris/lib/be/doris_be
18# doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:511
19# doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:482
20# doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:697
21# doris::vectorized::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader> > >&) at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:186
22# doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) at /root/tc/be/src/vec/olap/block_reader.cpp:139
23# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /root/tc/be/src/vec/olap/block_reader.cpp:211
24# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /root/tc/be/src/vec/exec/scan/new_olap_scanner.cpp:227
25# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /root/tc/be/src/vec/exec/scan/scanner_scheduler.cpp:259
26# doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#2}::operator()() const at /root/tc/be/src/vec/exec/scan/scanner_scheduler.cpp:180
...

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-01-07T08:33:36Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

fe/fe-core/src/main/java/org/apache/doris/planner/SelectNode.java

github-actions · 2025-01-07T11:31:34Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-01-07T11:31:36Z

PR approved by anyone and no changes requested.

liutang123 · 2025-01-07T11:34:03Z

run buildall

doris-robot · 2025-01-07T13:08:11Z

TPC-H: Total hot run time: 32931 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0f3bf56f7f8f46c106abbce71943ad89aab69e96, data reload: false

------ Round 1 ----------------------------------
q1	17601	6180	6053	6053
q2	2045	328	202	202
q3	10418	1271	754	754
q4	10248	900	452	452
q5	8048	2207	2027	2027
q6	213	182	146	146
q7	910	779	603	603
q8	9248	1406	1173	1173
q9	5323	4962	4967	4962
q10	6749	2277	1844	1844
q11	478	277	255	255
q12	343	359	219	219
q13	17763	3635	3089	3089
q14	247	236	217	217
q15	588	524	498	498
q16	624	609	592	592
q17	597	874	354	354
q18	7247	6378	6571	6378
q19	1877	971	555	555
q20	319	314	185	185
q21	2942	2262	2061	2061
q22	361	340	312	312
Total cold run time: 104189 ms
Total hot run time: 32931 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6333	6267	6232	6232
q2	229	324	230	230
q3	2219	2699	2347	2347
q4	1432	1835	1393	1393
q5	4340	4761	4924	4761
q6	193	174	139	139
q7	2133	2018	1829	1829
q8	2673	2794	2691	2691
q9	7313	7291	7400	7291
q10	3049	3343	2834	2834
q11	578	509	494	494
q12	652	712	607	607
q13	3567	3883	3272	3272
q14	302	317	285	285
q15	556	515	517	515
q16	662	690	663	663
q17	1244	1749	1245	1245
q18	7742	7482	7424	7424
q19	844	1111	1156	1111
q20	2083	2104	1921	1921
q21	5761	5152	5000	5000
q22	624	644	609	609
Total cold run time: 54529 ms
Total hot run time: 52893 ms

doris-robot · 2025-01-07T13:20:27Z

TPC-DS: Total hot run time: 196788 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0f3bf56f7f8f46c106abbce71943ad89aab69e96, data reload: false

query1	1313	972	928	928
query2	6290	2321	2328	2321
query3	11085	4949	4764	4764
query4	33174	23850	23414	23414
query5	3530	607	444	444
query6	258	194	175	175
query7	3993	491	299	299
query8	298	242	227	227
query9	9439	2780	2755	2755
query10	476	297	265	265
query11	17696	15330	15301	15301
query12	154	106	111	106
query13	1584	550	421	421
query14	9560	6809	9208	6809
query15	265	205	198	198
query16	7956	668	477	477
query17	1582	775	592	592
query18	1321	366	319	319
query19	204	185	177	177
query20	119	113	111	111
query21	208	123	105	105
query22	4695	4510	4645	4510
query23	34427	33648	34186	33648
query24	6519	2361	2355	2355
query25	535	455	389	389
query26	740	279	165	165
query27	2066	475	334	334
query28	5783	2542	2502	2502
query29	551	557	416	416
query30	209	182	153	153
query31	972	903	901	901
query32	101	64	63	63
query33	499	364	320	320
query34	779	877	519	519
query35	832	847	767	767
query36	1031	1051	976	976
query37	126	104	79	79
query38	4157	4124	4215	4124
query39	1540	1470	1517	1470
query40	213	126	110	110
query41	88	45	46	45
query42	114	101	104	101
query43	530	523	486	486
query44	1337	844	808	808
query45	184	182	170	170
query46	887	1060	665	665
query47	1940	2002	1936	1936
query48	398	407	329	329
query49	727	477	397	397
query50	653	683	399	399
query51	6999	7042	6980	6980
query52	104	100	122	100
query53	233	261	184	184
query54	472	528	413	413
query55	84	90	79	79
query56	266	269	254	254
query57	1235	1237	1166	1166
query58	267	225	221	221
query59	3127	3294	3040	3040
query60	274	277	248	248
query61	118	111	108	108
query62	897	835	753	753
query63	237	197	203	197
query64	3419	1013	679	679
query65	3307	3220	3322	3220
query66	721	418	314	314
query67	16512	15870	15562	15562
query68	8522	688	559	559
query69	488	313	258	258
query70	1186	1120	1136	1120
query71	438	279	256	256
query72	6346	3900	3881	3881
query73	659	758	360	360
query74	9892	9224	8910	8910
query75	3972	3167	2657	2657
query76	3703	1200	780	780
query77	764	371	289	289
query78	10117	10039	9442	9442
query79	3477	813	590	590
query80	713	525	429	429
query81	493	267	239	239
query82	576	153	130	130
query83	189	160	145	145
query84	283	95	75	75
query85	778	348	302	302
query86	344	319	304	304
query87	4394	4638	4523	4523
query88	3887	2250	2226	2226
query89	416	320	296	296
query90	1790	192	191	191
query91	133	138	105	105
query92	69	56	51	51
query93	1296	848	539	539
query94	634	398	305	305
query95	327	296	259	259
query96	516	617	281	281
query97	2909	3044	2806	2806
query98	224	205	190	190
query99	1707	1561	1427	1427
Total cold run time: 292652 ms
Total hot run time: 196788 ms

doris-robot · 2025-01-07T13:25:48Z

ClickBench: Total hot run time: 31.48 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0f3bf56f7f8f46c106abbce71943ad89aab69e96, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.06
query4	1.62	0.11	0.10
query5	0.41	0.41	0.42
query6	1.17	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.51	0.52
query10	0.57	0.57	0.54
query11	0.14	0.09	0.10
query12	0.14	0.11	0.12
query13	0.60	0.61	0.58
query14	2.73	2.86	2.75
query15	0.89	0.82	0.83
query16	0.37	0.37	0.38
query17	1.00	1.03	1.05
query18	0.23	0.22	0.21
query19	1.99	1.91	2.01
query20	0.01	0.01	0.01
query21	15.37	0.97	0.59
query22	0.73	0.77	0.69
query23	15.21	1.41	0.52
query24	3.43	1.73	0.96
query25	0.23	0.14	0.09
query26	0.32	0.16	0.13
query27	0.09	0.05	0.05
query28	13.50	1.51	1.05
query29	12.57	4.04	3.33
query30	0.25	0.09	0.07
query31	2.82	0.60	0.38
query32	3.22	0.54	0.47
query33	3.05	3.19	3.07
query34	16.73	5.15	4.51
query35	4.53	4.56	4.52
query36	0.86	0.50	0.49
query37	0.09	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.02	0.02
query40	0.16	0.12	0.12
query41	0.08	0.03	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.23 s
Total hot run time: 31.48 s

liutang123 · 2025-01-08T01:13:25Z

run p0

liutang123 · 2025-01-08T07:22:21Z

run buildall

doris-robot · 2025-01-08T07:43:52Z

TPC-H: Total hot run time: 32417 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 21eef3172b01966bccf3a220418792b0f7f52e70, data reload: false

------ Round 1 ----------------------------------
q1	17598	6096	6042	6042
q2	2051	289	160	160
q3	10442	1269	715	715
q4	10214	868	435	435
q5	7495	2159	1921	1921
q6	208	188	151	151
q7	882	741	604	604
q8	9239	1316	1113	1113
q9	5217	4906	4882	4882
q10	6783	2297	1831	1831
q11	466	276	279	276
q12	349	354	225	225
q13	17751	3694	3092	3092
q14	225	240	210	210
q15	558	514	502	502
q16	638	635	599	599
q17	562	839	325	325
q18	7169	6582	6323	6323
q19	1230	985	569	569
q20	311	329	198	198
q21	2832	2160	1946	1946
q22	356	327	298	298
Total cold run time: 102576 ms
Total hot run time: 32417 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6226	6281	7087	6281
q2	237	339	232	232
q3	2241	2654	2336	2336
q4	1357	1826	1374	1374
q5	4316	4696	4795	4696
q6	189	176	145	145
q7	2130	1960	1820	1820
q8	2622	2875	2638	2638
q9	7265	7148	7281	7148
q10	2996	3275	2826	2826
q11	612	511	496	496
q12	669	762	634	634
q13	3501	3861	3267	3267
q14	298	294	284	284
q15	561	509	510	509
q16	656	686	645	645
q17	1187	1741	1243	1243
q18	7812	7438	7150	7150
q19	795	1127	1054	1054
q20	1885	1956	1880	1880
q21	5465	5062	4782	4782
q22	595	633	596	596
Total cold run time: 53615 ms
Total hot run time: 52036 ms

doris-robot · 2025-01-08T07:55:46Z

TPC-DS: Total hot run time: 187963 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 21eef3172b01966bccf3a220418792b0f7f52e70, data reload: false

query1	961	382	369	369
query2	6527	2322	2312	2312
query3	6703	216	221	216
query4	33484	23487	23602	23487
query5	4332	630	447	447
query6	315	199	176	176
query7	4634	488	295	295
query8	285	234	216	216
query9	9764	2728	2721	2721
query10	472	329	249	249
query11	17809	15647	14980	14980
query12	150	103	108	103
query13	1680	542	417	417
query14	10663	6500	7559	6500
query15	255	186	195	186
query16	8777	580	474	474
query17	1639	772	583	583
query18	2123	397	297	297
query19	215	177	153	153
query20	119	112	112	112
query21	209	121	103	103
query22	4353	4324	4096	4096
query23	34190	34157	32873	32873
query24	6359	2224	2312	2224
query25	452	449	379	379
query26	1102	257	152	152
query27	2033	456	326	326
query28	5381	2464	2458	2458
query29	525	526	415	415
query30	229	186	146	146
query31	950	882	777	777
query32	101	60	60	60
query33	504	347	308	308
query34	746	830	502	502
query35	784	800	729	729
query36	1029	1033	952	952
query37	119	98	73	73
query38	4151	4050	4084	4050
query39	1478	1426	1419	1419
query40	196	116	101	101
query41	45	47	45	45
query42	120	103	104	103
query43	510	548	491	491
query44	1275	806	805	805
query45	174	165	161	161
query46	861	1041	633	633
query47	1817	1813	1779	1779
query48	381	404	317	317
query49	754	478	389	389
query50	621	647	385	385
query51	6984	6925	6739	6739
query52	104	98	93	93
query53	222	248	195	195
query54	477	481	408	408
query55	80	81	78	78
query56	255	266	247	247
query57	1141	1143	1066	1066
query58	236	228	225	225
query59	3037	3006	2850	2850
query60	285	272	247	247
query61	116	109	110	109
query62	867	757	709	709
query63	225	189	187	187
query64	4242	978	630	630
query65	3273	3150	3149	3149
query66	1059	411	305	305
query67	15897	15692	15471	15471
query68	8188	689	508	508
query69	458	288	258	258
query70	1187	1139	1137	1137
query71	426	280	259	259
query72	6134	3805	3888	3805
query73	657	758	364	364
query74	9596	8813	8919	8813
query75	3787	3161	2643	2643
query76	3610	1189	752	752
query77	767	353	270	270
query78	10062	9994	9872	9872
query79	3554	790	585	585
query80	635	507	437	437
query81	475	265	233	233
query82	546	148	120	120
query83	199	171	144	144
query84	290	91	70	70
query85	786	366	297	297
query86	403	314	291	291
query87	4365	4313	4254	4254
query88	4693	2222	2230	2222
query89	405	334	300	300
query90	1818	192	196	192
query91	137	140	105	105
query92	64	55	50	50
query93	1512	861	521	521
query94	662	380	282	282
query95	329	266	256	256
query96	488	604	279	279
query97	2895	2967	2816	2816
query98	233	203	198	198
query99	1563	1502	1378	1378
Total cold run time: 291780 ms
Total hot run time: 187963 ms

doris-robot · 2025-01-08T08:01:06Z

ClickBench: Total hot run time: 32.07 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 21eef3172b01966bccf3a220418792b0f7f52e70, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.61	0.12	0.10
query5	0.41	0.43	0.42
query6	1.16	0.65	0.65
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.57	0.51	0.52
query10	0.55	0.57	0.57
query11	0.14	0.10	0.10
query12	0.13	0.11	0.11
query13	0.61	0.62	0.61
query14	2.70	2.81	2.82
query15	0.90	0.83	0.83
query16	0.39	0.39	0.40
query17	1.04	1.06	1.04
query18	0.24	0.21	0.21
query19	1.98	1.88	1.99
query20	0.01	0.01	0.02
query21	15.36	0.95	0.58
query22	0.77	0.73	0.73
query23	15.27	1.39	0.55
query24	2.54	1.50	1.50
query25	0.15	0.18	0.07
query26	0.31	0.14	0.14
query27	0.05	0.06	0.05
query28	13.95	1.51	1.04
query29	12.59	3.98	3.31
query30	0.26	0.09	0.07
query31	2.81	0.58	0.38
query32	3.23	0.53	0.46
query33	3.12	3.09	3.05
query34	16.78	5.13	4.52
query35	4.51	4.48	4.43
query36	0.67	0.48	0.50
query37	0.09	0.06	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.12
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.69 s
Total hot run time: 32.07 s

github-actions · 2025-01-08T11:05:26Z

PR approved by at least one committer and no changes requested.

…#46541) ### What problem does this PR solve? Problem Summary: In 2.1.7, if a sql parse failed in nereids planner, it will fallback to old planner. and old planner maybe create `SelectNode` with some conjuncts that are not boolean type. Reproduce: Create table sql see #46498 query sql1: select b.c_id from dbgr as b left join (select c.c_id from lo where event_date between 20220500 and 20220600 limit 100 )c on c.c_id LIMIT 200; query sql2: select b.c_id from dbgr as b left join (select c.c_id from lo )c on c.c_id LIMIT 0, 200; Because `select c.c_id`, these sqls will fallback to old planner. Because `on c.c_id` is not boolean type, and be will core. A part of query plan is as follows: | 1:VOlapScanNode | | TABLE: test.lo(lo), PREAGGREGATION: ON | | PREDICATES: (`c`.`c_id` AND (`test`.`lo`.`__DORIS_DELETE_SIGN__` = 0)) | | partitions=1/3 (p_202206) | | tablets=2/2, tabletList=89678,89680 | | cardinality=46, avgRowSize=165.54349, numNodes=1 | | pushAggOp=NONE | +-----------------------------------------------------------------------------+ A fatal log is as follows: F20241219 23:13:23.457937 33282 assert_cast.h:58] Bad cast from type:doris::vectorized::ColumnVector<int> to doris::vectorized::ColumnVector<unsigned char> *** Check failure stack trace: *** @ 0x55bfa043b956 google::LogMessageFatal::~LogMessageFatal() @ 0x55bf6f3bc070 assert_cast<>() @ 0x55bf8978d767 doris::vectorized::VExprContext::execute_conjuncts() @ 0x55bf8978c463 doris::vectorized::VExprContext::execute_conjuncts_and_filter_block() @ 0x55bf8978bf72 doris::vectorized::VExprContext::filter_block() @ 0x55bfa035b8e4 doris::pipeline::SelectOperatorX::pull() @ 0x55bf9fee2b62 doris::pipeline::StreamingOperatorX<>::get_block() @ 0x55bf9feab54b doris::pipeline::OperatorXBase::get_block_after_projects() @ 0x55bfa03dd07c doris::pipeline::PipelineXTask::execute() @ 0x55bfa0413e85 doris::pipeline::TaskScheduler::_do_work() @ 0x55bfa0417dcb doris::pipeline::TaskScheduler::start()::$_0::operator()() @ 0x55bfa0417d55 std::__invoke_impl<>() @ 0x55bfa0417d05 _ZSt10__invoke_rIvRZN5doris8pipeline13TaskScheduler5startEvE3$_0JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES6_E4typeEO S7_DpOS8_ @ 0x55bfa0417bcd std::_Function_handler<>::_M_invoke() @ 0x55bf6e6c9b63 std::function<>::operator()() @ 0x55bf7289e209 doris::FunctionRunnable::run() @ 0x55bf728899c0 doris::ThreadPool::dispatch_thread() @ 0x55bf728b0c24 std::__invoke_impl<>() @ 0x55bf728b0afd std::__invoke<>() @ 0x55bf728b0a85 _ZNSt5_BindIFMN5doris10ThreadPoolEFvvEPS1_EE6__callIvJEJLm0EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE @ 0x55bf728b092e std::_Bind<>::operator()<>() @ 0x55bf728b0845 std::__invoke_impl<>() @ 0x55bf728b07e5 _ZSt10__invoke_rIvRSt5_BindIFMN5doris10ThreadPoolEFvvEPS2_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESA_E4typeEOSB_D pOSC_ @ 0x55bf728b048d std::_Function_handler<>::_M_invoke() @ 0x55bf6e6c9b63 std::function<>::operator()() @ 0x55bf728521fc doris::Thread::supervise_thread() @ 0x7f4260614ea5 start_thread @ 0x7f42610439fd __clone @ (nil) (unknown) And another: F20250108 13:07:05.275424 184257 assert_cast.h:58] Bad cast from type:doris::vectorized::ColumnVector<int> to doris::vectorized::ColumnVector<unsigned char> 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/tc/be/src/common/signal_handler.h:421 1# 0x00007FB73FB31400 in /lib64/libc.so.6 2# __GI_raise in /lib64/libc.so.6 3# abort in /lib64/libc.so.6 4# 0x000055CDAAF0090D in /usr/local/service/doris/lib/be/doris_be 5# google::LogMessage::SendToLog() in /usr/local/service/doris/lib/be/doris_be 6# google::LogMessage::Flush() in /usr/local/service/doris/lib/be/doris_be 7# google::LogMessageFatal::~LogMessageFatal() in /usr/local/service/doris/lib/be/doris_be 8# doris::vectorized::ColumnVector<unsigned char> const& assert_cast<doris::vectorized::ColumnVector<unsigned char> const&, doris::vec torized::IColumn const&>(doris::vectorized::IColumn const&) in /usr/local/service/doris/lib/be/doris_be 9# doris::vectorized::VExprContext::execute_conjuncts(std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std ::shared_ptr<doris::vectorized::VExprContext> > > const&, std::vector<doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<fals e, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*, std::allocator<doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<fal se, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*> > const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray<unsigne d char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*, bool*) at /root/tc/be/src/vec/exprs/vexpr_context .cpp:181 10# doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector<std::shared_ptr<doris::vectorized::VExprContext>, s td::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const&, doris::vectorized::Block*, std::vector<unsigned int, std::al locator<unsigned int> >&, int, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator >, 16ul, 16ul>&) at /root/tc/be/src/vec/exprs/vexpr_context.cpp:324 11# doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) at /root/tc/b e/src/olap/rowset/segment_v2/segment_iterator.cpp:2274 12# doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2178 13# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1914 14# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1913 15# doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) in /usr/local/service/doris/lib/be/doris_be 16# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/beta_rowset_reader.cpp:348 17# doris::vectorized::VCollectIterator::Level0Iterator::_refresh() in /usr/local/service/doris/lib/be/doris_be 18# doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:511 19# doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:482 20# doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:697 21# doris::vectorized::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader> > >&) at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:186 22# doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) at /root/tc/be/src/vec/olap/block_reader.cpp:139 23# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /root/tc/be/src/vec/olap/block_reader.cpp:211 24# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /root/tc/be/src/vec/exec/scan/new_olap_scanner.cpp:227 25# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /root/tc/be/src/vec/exec/scan/scanner_scheduler.cpp:259 26# doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#2}::operator()() const at /root/tc/be/src/vec/exec/scan/scanner_scheduler.cpp:180 ... --------- Co-authored-by: liutang123 <[email protected]>

… old planner #46541 (#46638) Cherry-picked from #46541 Co-authored-by: Lijia Liu <[email protected]> Co-authored-by: liutang123 <[email protected]>

… old planner #46541 (#46637) Cherry-picked from #46541 Co-authored-by: Lijia Liu <[email protected]> Co-authored-by: liutang123 <[email protected]>

liutang123 added the dev/2.1.x label Jan 7, 2025

morrySnow reviewed Jan 7, 2025

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/planner/SelectNode.java Outdated Show resolved Hide resolved

morrySnow previously approved these changes Jan 7, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 7, 2025

github-actions bot added the reviewed label Jan 7, 2025

liutang123 dismissed morrySnow’s stale review via 0bc2d47 January 8, 2025 07:03

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Jan 8, 2025

liutang123 added 2 commits January 8, 2025 15:04

[fix](planner) select and filter by slot in old planner

65ada34

change cast PREDICATES logic to PlanNode

21eef31

liutang123 force-pushed the fix-old-planner-select-slot branch from 0bc2d47 to 21eef31 Compare January 8, 2025 07:04

liutang123 added the dev/3.0.x label Jan 8, 2025

cambyzju approved these changes Jan 8, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 8, 2025

morrySnow approved these changes Jan 8, 2025

View reviewed changes

morrySnow merged commit 8405321 into apache:master Jan 8, 2025
25 of 28 checks passed

github-actions bot mentioned this pull request Jan 8, 2025

branch-3.0: [fix](planner) fix core when select and filter by slot in old planner #46541 #46637

Merged

github-actions bot mentioned this pull request Jan 8, 2025

branch-2.1: [fix](planner) fix core when select and filter by slot in old planner #46541 #46638

Merged

yiguolei added dev/2.1.8-merged and removed dev/2.1.x labels Jan 8, 2025

yiguolei added dev/3.0.4-merged usercase Important user case type label and removed dev/3.0.x labels Jan 8, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](planner) fix core when select and filter by slot in old planner #46541

[fix](planner) fix core when select and filter by slot in old planner #46541

liutang123 commented Jan 7, 2025 •

edited

Loading

hello-stephen commented Jan 7, 2025

github-actions bot commented Jan 7, 2025

github-actions bot commented Jan 7, 2025

liutang123 commented Jan 7, 2025

doris-robot commented Jan 7, 2025

doris-robot commented Jan 7, 2025

doris-robot commented Jan 7, 2025

liutang123 commented Jan 8, 2025

liutang123 commented Jan 8, 2025

doris-robot commented Jan 8, 2025

doris-robot commented Jan 8, 2025

doris-robot commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

[fix](planner) fix core when select and filter by slot in old planner #46541

[fix](planner) fix core when select and filter by slot in old planner #46541

Conversation

liutang123 commented Jan 7, 2025 • edited Loading

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

hello-stephen commented Jan 7, 2025

github-actions bot commented Jan 7, 2025

github-actions bot commented Jan 7, 2025

liutang123 commented Jan 7, 2025

doris-robot commented Jan 7, 2025

doris-robot commented Jan 7, 2025

doris-robot commented Jan 7, 2025

liutang123 commented Jan 8, 2025

liutang123 commented Jan 8, 2025

doris-robot commented Jan 8, 2025

doris-robot commented Jan 8, 2025

doris-robot commented Jan 8, 2025

github-actions bot commented Jan 8, 2025

liutang123 commented Jan 7, 2025 •

edited

Loading