branch-3.0: [fix](planner) fix core when select and filter by slot in old planner #46541 #46637
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
…#46541) ### What problem does this PR solve? Problem Summary: In 2.1.7, if a sql parse failed in nereids planner, it will fallback to old planner. and old planner maybe create `SelectNode` with some conjuncts that are not boolean type. Reproduce: Create table sql see #46498 query sql1: select b.c_id from dbgr as b left join (select c.c_id from lo where event_date between 20220500 and 20220600 limit 100 )c on c.c_id LIMIT 200; query sql2: select b.c_id from dbgr as b left join (select c.c_id from lo )c on c.c_id LIMIT 0, 200; Because `select c.c_id`, these sqls will fallback to old planner. Because `on c.c_id` is not boolean type, and be will core. A part of query plan is as follows: | 1:VOlapScanNode | | TABLE: test.lo(lo), PREAGGREGATION: ON | | PREDICATES: (`c`.`c_id` AND (`test`.`lo`.`__DORIS_DELETE_SIGN__` = 0)) | | partitions=1/3 (p_202206) | | tablets=2/2, tabletList=89678,89680 | | cardinality=46, avgRowSize=165.54349, numNodes=1 | | pushAggOp=NONE | +-----------------------------------------------------------------------------+ A fatal log is as follows: F20241219 23:13:23.457937 33282 assert_cast.h:58] Bad cast from type:doris::vectorized::ColumnVector<int> to doris::vectorized::ColumnVector<unsigned char> *** Check failure stack trace: *** @ 0x55bfa043b956 google::LogMessageFatal::~LogMessageFatal() @ 0x55bf6f3bc070 assert_cast<>() @ 0x55bf8978d767 doris::vectorized::VExprContext::execute_conjuncts() @ 0x55bf8978c463 doris::vectorized::VExprContext::execute_conjuncts_and_filter_block() @ 0x55bf8978bf72 doris::vectorized::VExprContext::filter_block() @ 0x55bfa035b8e4 doris::pipeline::SelectOperatorX::pull() @ 0x55bf9fee2b62 doris::pipeline::StreamingOperatorX<>::get_block() @ 0x55bf9feab54b doris::pipeline::OperatorXBase::get_block_after_projects() @ 0x55bfa03dd07c doris::pipeline::PipelineXTask::execute() @ 0x55bfa0413e85 doris::pipeline::TaskScheduler::_do_work() @ 0x55bfa0417dcb doris::pipeline::TaskScheduler::start()::$_0::operator()() @ 0x55bfa0417d55 std::__invoke_impl<>() @ 0x55bfa0417d05 _ZSt10__invoke_rIvRZN5doris8pipeline13TaskScheduler5startEvE3$_0JEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EES6_E4typeEO S7_DpOS8_ @ 0x55bfa0417bcd std::_Function_handler<>::_M_invoke() @ 0x55bf6e6c9b63 std::function<>::operator()() @ 0x55bf7289e209 doris::FunctionRunnable::run() @ 0x55bf728899c0 doris::ThreadPool::dispatch_thread() @ 0x55bf728b0c24 std::__invoke_impl<>() @ 0x55bf728b0afd std::__invoke<>() @ 0x55bf728b0a85 _ZNSt5_BindIFMN5doris10ThreadPoolEFvvEPS1_EE6__callIvJEJLm0EEEET_OSt5tupleIJDpT0_EESt12_Index_tupleIJXspT1_EEE @ 0x55bf728b092e std::_Bind<>::operator()<>() @ 0x55bf728b0845 std::__invoke_impl<>() @ 0x55bf728b07e5 _ZSt10__invoke_rIvRSt5_BindIFMN5doris10ThreadPoolEFvvEPS2_EEJEENSt9enable_ifIX16is_invocable_r_vIT_T0_DpT1_EESA_E4typeEOSB_D pOSC_ @ 0x55bf728b048d std::_Function_handler<>::_M_invoke() @ 0x55bf6e6c9b63 std::function<>::operator()() @ 0x55bf728521fc doris::Thread::supervise_thread() @ 0x7f4260614ea5 start_thread @ 0x7f42610439fd __clone @ (nil) (unknown) And another: F20250108 13:07:05.275424 184257 assert_cast.h:58] Bad cast from type:doris::vectorized::ColumnVector<int> to doris::vectorized::ColumnVector<unsigned char> 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/tc/be/src/common/signal_handler.h:421 1# 0x00007FB73FB31400 in /lib64/libc.so.6 2# __GI_raise in /lib64/libc.so.6 3# abort in /lib64/libc.so.6 4# 0x000055CDAAF0090D in /usr/local/service/doris/lib/be/doris_be 5# google::LogMessage::SendToLog() in /usr/local/service/doris/lib/be/doris_be 6# google::LogMessage::Flush() in /usr/local/service/doris/lib/be/doris_be 7# google::LogMessageFatal::~LogMessageFatal() in /usr/local/service/doris/lib/be/doris_be 8# doris::vectorized::ColumnVector<unsigned char> const& assert_cast<doris::vectorized::ColumnVector<unsigned char> const&, doris::vec torized::IColumn const&>(doris::vectorized::IColumn const&) in /usr/local/service/doris/lib/be/doris_be 9# doris::vectorized::VExprContext::execute_conjuncts(std::vector<std::shared_ptr<doris::vectorized::VExprContext>, std::allocator<std ::shared_ptr<doris::vectorized::VExprContext> > > const&, std::vector<doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<fals e, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*, std::allocator<doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<fal se, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*> > const*, bool, doris::vectorized::Block*, doris::vectorized::PODArray<unsigne d char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator>, 16ul, 16ul>*, bool*) at /root/tc/be/src/vec/exprs/vexpr_context .cpp:181 10# doris::vectorized::VExprContext::execute_conjuncts_and_filter_block(std::vector<std::shared_ptr<doris::vectorized::VExprContext>, s td::allocator<std::shared_ptr<doris::vectorized::VExprContext> > > const&, doris::vectorized::Block*, std::vector<unsigned int, std::al locator<unsigned int> >&, int, doris::vectorized::PODArray<unsigned char, 4096ul, Allocator<false, false, false, DefaultMemoryAllocator >, 16ul, 16ul>&) at /root/tc/be/src/vec/exprs/vexpr_context.cpp:324 11# doris::segment_v2::SegmentIterator::_execute_common_expr(unsigned short*, unsigned short&, doris::vectorized::Block*) at /root/tc/b e/src/olap/rowset/segment_v2/segment_iterator.cpp:2274 12# doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2178 13# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1914 14# doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1913 15# doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) in /usr/local/service/doris/lib/be/doris_be 16# doris::BetaRowsetReader::next_block(doris::vectorized::Block*) at /root/tc/be/src/olap/rowset/beta_rowset_reader.cpp:348 17# doris::vectorized::VCollectIterator::Level0Iterator::_refresh() in /usr/local/service/doris/lib/be/doris_be 18# doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:511 19# doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:482 20# doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:697 21# doris::vectorized::VCollectIterator::build_heap(std::vector<std::shared_ptr<doris::RowsetReader>, std::allocator<std::shared_ptr<doris::RowsetReader> > >&) at /root/tc/be/src/vec/olap/vcollect_iterator.cpp:186 22# doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) at /root/tc/be/src/vec/olap/block_reader.cpp:139 23# doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) at /root/tc/be/src/vec/olap/block_reader.cpp:211 24# doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) at /root/tc/be/src/vec/exec/scan/new_olap_scanner.cpp:227 25# doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) at /root/tc/be/src/vec/exec/scan/scanner_scheduler.cpp:259 26# doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#2}::operator()() const at /root/tc/be/src/vec/exec/scan/scanner_scheduler.cpp:180 ... --------- Co-authored-by: liutang123 <[email protected]>
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
|
run buildall |
TPC-H: Total hot run time: 40759 ms |
TPC-DS: Total hot run time: 198557 ms |
ClickBench: Total hot run time: 32.17 s |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Cherry-picked from #46541