[Bug](scan) do not release tablet_reader on NewOlapScanner::close (#46296)

BiteTheDDDDt · web-flow · commit 73800052bc87 · 2025-01-03T13:52:45.000+08:00
### What problem does this PR solve? ```cpp 172.20.50.85 be.out: #8 0x56435579ab5e in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:192:9 #9 0x564319f7d95d in doris::segment_v2::CSIndexInput::readInternal(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_compound_reader.cpp:107:11 #10 0x56435579bb5b in lucene::store::BufferedIndexInput::refill() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:285:5 #11 0x56435579b096 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int, int, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:217:9 #12 0x56435579a9f6 in lucene::store::BufferedIndexInput::readBytes(unsigned char*, int) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/clucene/src/core/CLucene/store/IndexInput.cpp:186:9 #13 0x564319fe8aee in doris::segment_v2::InvertedIndexReader::read_null_bitmap(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::segment_v2::InvertedIndexQueryCacheHandle*, lucene::store::Directory*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:144:29 #14 0x564319febab4 in doris::segment_v2::InvertedIndexReader::handle_searcher_cache(doris::segment_v2::InvertedIndexCacheHandle*, doris::io::IOContext const*, doris::OlapReaderStatistics*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:194:27 #15 0x564319ff4b0b in doris::segment_v2::StringTypeInvertedIndexReader::query(doris::io::IOContext const*, doris::OlapReaderStatistics*, doris::RuntimeState*, std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, std::shared_ptr&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:407:5 #16 0x56431a001f00 in doris::segment_v2::InvertedIndexIterator::read_from_inverted_index(std::__cxx11::basic_string, std::allocator> const&, void const*, doris::segment_v2::InvertedIndexQueryType, unsigned int, std::shared_ptr&, bool) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/inverted_index_reader.cpp:1137:5 #17 0x5643191f4a52 in doris::ComparisonPredicateBase<(doris::PrimitiveType)23, (doris::PredicateType)1>::evaluate(std::pair, std::allocator>, std::shared_ptr> const&, doris::segment_v2::InvertedIndexIterator*, unsigned int, roaring::Roaring*) const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/comparison_predicate.h:107:9 #18 0x56431a249ef7 in doris::segment_v2::SegmentIterator::_apply_inverted_index_on_column_predicate(doris::ColumnPredicate*, std::vector>&, bool*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:871:28 #19 0x56431a23ddc0 in doris::segment_v2::SegmentIterator::_apply_inverted_index() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:951:13 #20 0x56431a22f956 in doris::segment_v2::SegmentIterator::_get_row_ranges_by_column_conditions() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:512:13 #21 0x56431a22b78f in doris::segment_v2::SegmentIterator::_lazy_init() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:383:5 #22 0x56431a26c667 in doris::segment_v2::SegmentIterator::_next_batch_internal(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2008:9 #23 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1917:9 #24 0x56431a264fb6 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1916:19 #25 0x56431a08315c in doris::segment_v2::LazyInitSegmentIterator::next_batch(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44:33 #26 0x564319b95855 in doris::BetaRowsetReader::next_block(doris::vectorized::Block*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/olap/rowset/beta_rowset_reader.cpp:366:29 #27 0x56434e3a0c10 in doris::vectorized::VCollectIterator::Level0Iterator::_refresh() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.h #28 0x56434e38ba5b in doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:509:24 #29 0x56434e38cd76 in doris::vectorized::VCollectIterator::Level0Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:482:14 #30 0x56434e399745 in doris::vectorized::VCollectIterator::Level1Iterator::ensure_first_row_ref() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:696:27 #31 0x56434e37f7f9 in doris::vectorized::VCollectIterator::build_heap(std::vector, std::allocator>>&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/vcollect_iterator.cpp:186:9 #32 0x56434e312fe2 in doris::vectorized::BlockReader::_init_collect_iter(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:140:5 #33 0x56434e3165df in doris::vectorized::BlockReader::init(doris::TabletReader::ReaderParams const&) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/olap/block_reader.cpp:212:19 #34 0x5643515aba04 in doris::vectorized::NewOlapScanner::open(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:231:32 #35 0x564336c138c6 in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr, std::shared_ptr) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:247:5 #36 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const::'lambda0'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:180:21 #37 0x564336c18db0 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()::operator()() const /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:179:31 #38 0x564336c18db0 in void std::__invoke_impl, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61:14 #39 0x564336c18db0 in std::enable_if, std::shared_ptr)::$_1::operator()() const::'lambda'()&>, void>::type std::__invoke_r, std::shared_ptr)::$_1::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr, std::shared_ptr)::$_1::operator()() const::'lambda'()&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111:2 #40 0x564336c18db0 in std::_Function_handler, std::shared_ptr)::$_1::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291:9 #41 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24 #42 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5 #43 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8 #44 0x7f31d3c1984f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 0x61d000a7f3c8 is located 1352 bytes inside of 2064-byte region [0x61d000a7ee80,0x61d000a7f690) freed by thread T1698 (Pipe_normal [wo) here: #0 0x564317278d9d in operator delete(void*) (/mnt/hdd01/ci/doris-deploy-branch-3.0-cloud/be/lib/doris_be+0x3626bd9d) (BuildId: 9dab40b94b1dc995) #1 0x5643515b192c in std::unique_ptr>::reset(doris::TabletReader*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:456:7 #2 0x5643515b192c in doris::vectorized::NewOlapScanner::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:534:20 #3 0x564352457463 in doris::vectorized::ScannerDelegate::~ScannerDelegate() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/exec/scan/vscan_node.h:35:31 #4 0x5643172a8324 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:168:6 #5 0x56434e7d7870 in std::__shared_ptr::~__shared_ptr() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base.h:1149:31 #6 0x56434e7d7870 in void std::destroy_at>(std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_construct.h:88:15 #7 0x56434e7d7870 in void std::allocator_traits>>>::destroy>(std::allocator>>&, std::shared_ptr*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/alloc_traits.h:533:4 #8 0x56434e7d7870 in std::__cxx11::_List_base, std::allocator>>::_M_clear() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/list.tcc:77:4 #9 0x564351db11d2 in std::__cxx11::_List_base, std::allocator>>::~_List_base() /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_list.h:499:9 #10 0x564351db11d2 in doris::pipeline::ScanLocalState::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/scan_operator.cpp:1261:5 #11 0x56434e6357e3 in doris::pipeline::OperatorXBase::close(doris::RuntimeState*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/exec/operator.cpp:245:28 #12 0x564352ad00c1 in doris::pipeline::PipelineTask::close(doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/pipeline_task.cpp:489:28 #13 0x564352b0701f in doris::pipeline::_close_task(doris::pipeline::PipelineTask*, doris::Status) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:91:27 #14 0x564352b095f3 in doris::pipeline::TaskScheduler::_do_work(unsigned long) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/pipeline/task_scheduler.cpp:181:17 #15 0x56431b9d94eb in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:543:24 #16 0x56431b9b18c7 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/thread.cpp:498:5 #17 0x7f31d3b87ac2 in start_thread nptl/pthread_create.c:442:8 ``` Problem Summary: ### Release note None ### Check List (For Author) - Test  - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason  - Behavior changed: - [ ] No. - [ ] Yes.  - Does this need documentation? - [ ] No. - [ ] Yes.  ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label
diff --git a/be/src/vec/exec/scan/new_olap_scanner.cpp b/be/src/vec/exec/scan/new_olap_scanner.cpp
@@ -524,14 +524,6 @@ Status NewOlapScanner::close(RuntimeState* state) {
         return Status::OK();
     }
 
-    // olap scan node will call scanner.close() when finished
-    // will release resources here
-    // if not clear rowset readers in read_params here
-    // readers will be release when runtime state deconstructed but
-    // deconstructor in reader references runtime state
-    // so that it will core
-    _tablet_reader_params.rs_splits.clear();
-    _tablet_reader.reset();
     RETURN_IF_ERROR(VScanner::close(state));
     return Status::OK();
 }

Original file line number	Diff line number	Diff line change
`@@ -524,14 +524,6 @@ Status NewOlapScanner::close(RuntimeState* state) {`
`524`	`524`	`return Status::OK();`
`525`	`525`	`}`
`526`	`526`
`527`		`- // olap scan node will call scanner.close() when finished`
`528`		`- // will release resources here`
`529`		`- // if not clear rowset readers in read_params here`
`530`		`- // readers will be release when runtime state deconstructed but`
`531`		`- // deconstructor in reader references runtime state`
`532`		`- // so that it will core`
`533`		`- _tablet_reader_params.rs_splits.clear();`
`534`		`- _tablet_reader.reset();`
`535`	`527`	`RETURN_IF_ERROR(VScanner::close(state));`
`536`	`528`	`return Status::OK();`
`537`	`529`	`}`