[Refactor](Variant) refactor flush logic to support partial update #34925
Conversation
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
| } | ||
|
|
||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column) { | ||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column, |
There was a problem hiding this comment.
warning: function '_create_column_writer' has cognitive complexity of 51 (threshold 50) [readability-function-cognitive-complexity]
Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:154: +1, including nesting penalty of 0, nesting level increased to 1
if (tablet_index) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:163: +1, including nesting penalty of 0, nesting level increased to 1
if (_opts.rowset_ctx != nullptr) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:168: +1, including nesting penalty of 0, nesting level increased to 1
if (_opts.write_type == DataWriteType::TYPE_DIRECT &&
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:174: +1, including nesting penalty of 0, nesting level increased to 1
if (!InvertedIndexColumnWriter::check_column_valid(column)) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:213: +1, including nesting penalty of 0, nesting level increased to 1
if (column.is_row_store_column()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:218: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(ColumnWriter::create(opts, &column, _file_writer, &writer));
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:218: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(ColumnWriter::create(opts, &column, _file_writer, &writer));
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:219: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(writer->init());
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:219: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(writer->init());
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^| } | ||
|
|
||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column) { | ||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column, |
There was a problem hiding this comment.
warning: function '_create_column_writer' exceeds recommended size/complexity thresholds [readability-function-size]
Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:142: 81 lines including whitespace and comments (threshold 80)
Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column,
^| @@ -308,13 +305,14 @@ | |||
| // 2.2 build read plan to read by batch | |||
| // 2.3 fill block | |||
| // 3. set columns to data convertor and then write all columns | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data) { | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data, | |||
There was a problem hiding this comment.
warning: function '_append_block_with_partial_content' has cognitive complexity of 57 (threshold 50) [readability-function-cognitive-complexity]
Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:351: +1, including nesting penalty of 0, nesting level increased to 1
if (const vectorized::ColumnWithTypeAndName* delete_sign_column =
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:356: +2, including nesting penalty of 1, nesting level increased to 2
if (delete_sign_col.size() >= data.row_pos + data.num_rows) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:363: +1, including nesting penalty of 0, nesting level increased to 1
DBUG_EXECUTE_IF("VerticalSegmentWriter._append_block_with_partial_content.sleep",
^be/src/util/debug_points.h:35: expanded from macro 'DBUG_EXECUTE_IF'
if (UNLIKELY(config::enable_debug_points)) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:363: +2, including nesting penalty of 1, nesting level increased to 2
DBUG_EXECUTE_IF("VerticalSegmentWriter._append_block_with_partial_content.sleep",
^be/src/util/debug_points.h:37: expanded from macro 'DBUG_EXECUTE_IF'
if (dp) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:367: +1, including nesting penalty of 0, nesting level increased to 1
if (specified_rowsets.size() != _mow_context->rowset_ids.size()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:376: +2, including nesting penalty of 1, nesting level increased to 2
if (_opts.rowset_ctx->partial_update_info->is_strict_mode) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:387: +1, including nesting penalty of 0, nesting level increased to 1
for (size_t block_pos = data.row_pos; block_pos < data.row_pos + data.num_rows; block_pos++) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:398: +2, including nesting penalty of 1, nesting level increased to 2
if (have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:404: +2, including nesting penalty of 1, nesting level increased to 2
if (!_tablet_schema->has_sequence_col() || have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:405: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:405: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:410: +1
(delete_sign_column_data != nullptr && delete_sign_column_data[block_pos] != 0);
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:417: +2, including nesting penalty of 1, nesting level increased to 2
if (st.is<KEY_NOT_FOUND>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:418: +3, including nesting penalty of 2, nesting level increased to 3
if (_opts.rowset_ctx->partial_update_info->is_strict_mode) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:424: +1, nesting level increased to 3
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:425: +4, including nesting penalty of 3, nesting level increased to 4
if (!_opts.rowset_ctx->partial_update_info->can_insert_new_rows_in_partial_update) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:446: +2, including nesting penalty of 1, nesting level increased to 2
if (!st.ok() && !st.is<KEY_ALREADY_EXISTS>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:456: +2, including nesting penalty of 1, nesting level increased to 2
if (have_delete_sign && !_tablet_schema->has_sequence_col()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:459: +1, nesting level increased to 2
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:466: +2, including nesting penalty of 1, nesting level increased to 2
if (st.is<KEY_ALREADY_EXISTS>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:472: +1, nesting level increased to 2
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:479: +1, including nesting penalty of 0, nesting level increased to 1
if (config::enable_merge_on_write_correctness_check) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:486: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(_fill_missing_columns(mutable_full_columns, use_default_or_null_flag,
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:486: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(_fill_missing_columns(mutable_full_columns, use_default_or_null_flag,
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:489: +1, including nesting penalty of 0, nesting level increased to 1
if (_tablet_schema->store_row_column()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:513: +1, including nesting penalty of 0, nesting level increased to 1
if (_tablet_schema->has_sequence_col() && !have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:520: +2, including nesting penalty of 1, nesting level increased to 2
if (_num_rows_written != data.row_pos ||
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:527: +2, including nesting penalty of 1, nesting level increased to 2
for (size_t block_pos = data.row_pos; block_pos < data.row_pos + data.num_rows;
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:531: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:531: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^| @@ -308,13 +305,14 @@ | |||
| // 2.2 build read plan to read by batch | |||
| // 2.3 fill block | |||
| // 3. set columns to data convertor and then write all columns | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data) { | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data, | |||
There was a problem hiding this comment.
warning: function '_append_block_with_partial_content' exceeds recommended size/complexity thresholds [readability-function-size]
Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:307: 233 lines including whitespace and comments (threshold 80)
Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data,
^| @@ -709,16 +707,122 @@ | |||
| return Status::OK(); | |||
| } | |||
|
|
|||
| Status VerticalSegmentWriter::_append_block_with_variant_subcolumns(RowsInBlock& data) { | |||
There was a problem hiding this comment.
warning: function '_append_block_with_variant_subcolumns' exceeds recommended size/complexity thresholds [readability-function-size]
Status VerticalSegmentWriter::_append_block_with_variant_subcolumns(RowsInBlock& data) {
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:709: 98 lines including whitespace and comments (threshold 80)
Status VerticalSegmentWriter::_append_block_with_variant_subcolumns(RowsInBlock& data) {
^
TPC-H: Total hot run time: 40785 ms |
TPC-DS: Total hot run time: 187021 ms |
ClickBench: Total hot run time: 31.03 s |
|
TeamCity be ut coverage result: |
eldenmoon
force-pushed
the
var-rf-seg
branch
3 times, most recently
from
d7fd687 to
826e2c9
Compare
|
run buildall |
eldenmoon
changed the title
|
TeamCity be ut coverage result: |
|
run buildall |
|
|
||
| Status parse_variant_columns(Block& block, const std::vector<int>& variant_pos, | ||
| const ParseContext& ctx) { | ||
| Status _parse_variant_columns(Block& block, const std::vector<int>& variant_pos, |
There was a problem hiding this comment.
warning: function '_parse_variant_columns' has cognitive complexity of 59 (threshold 50) [readability-function-cognitive-complexity]
Status _parse_variant_columns(Block& block, const std::vector<int>& variant_pos,
^Additional context
be/src/vec/common/schema_util.cpp:481: +1, including nesting penalty of 0, nesting level increased to 1
for (int i = 0; i < variant_pos.size(); ++i) {
^be/src/vec/common/schema_util.cpp:491: nesting level increased to 2
auto encode_rowstore = [&]() {
^be/src/vec/common/schema_util.cpp:492: +3, including nesting penalty of 2, nesting level increased to 3
if (!ctx.record_raw_json_column) {
^be/src/vec/common/schema_util.cpp:496: +3, including nesting penalty of 2, nesting level increased to 3
if (record_raw_string_with_serialization) {
^be/src/vec/common/schema_util.cpp:499: +4, including nesting penalty of 3, nesting level increased to 4
for (size_t i = 0; i < var->rows(); ++i) {
^be/src/vec/common/schema_util.cpp:501: +5, including nesting penalty of 4, nesting level increased to 5
RETURN_IF_ERROR(var->serialize_one_row_to_string(i, &raw_str));
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:501: +6, including nesting penalty of 5, nesting level increased to 6
RETURN_IF_ERROR(var->serialize_one_row_to_string(i, &raw_str));
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:505: +1, nesting level increased to 3
} else {
^be/src/vec/common/schema_util.cpp:515: +2, including nesting penalty of 1, nesting level increased to 2
if (!var.is_scalar_variant()) {
^be/src/vec/common/schema_util.cpp:518: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:518: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:523: +2, including nesting penalty of 1, nesting level increased to 2
if (WhichDataType(remove_nullable(var.get_root_type())).is_json()) {
^be/src/vec/common/schema_util.cpp:526: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(cast_column({var.get_root(), var.get_root_type(), ""},
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:528: +4, including nesting penalty of 3, nesting level increased to 4
? make_nullable(std::make_shared<DataTypeString>())
^be/src/vec/common/schema_util.cpp:526: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(cast_column({var.get_root(), var.get_root_type(), ""},
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:531: +3, including nesting penalty of 2, nesting level increased to 3
if (raw_json_column->is_nullable()) {
^be/src/vec/common/schema_util.cpp:535: +1, nesting level increased to 2
} else {
^be/src/vec/common/schema_util.cpp:539: +3, including nesting penalty of 2, nesting level increased to 3
? assert_cast<const ColumnNullable&>(root).get_nested_column_ptr()
^be/src/vec/common/schema_util.cpp:549: +2, including nesting penalty of 1, nesting level increased to 2
if (is_nullable) {
^be/src/vec/common/schema_util.cpp:555: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:570: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:555: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:572: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^
|
TeamCity be ut coverage result: |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
| } | ||
|
|
||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column) { | ||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column, |
There was a problem hiding this comment.
warning: function '_create_column_writer' has cognitive complexity of 51 (threshold 50) [readability-function-cognitive-complexity]
Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:154: +1, including nesting penalty of 0, nesting level increased to 1
if (tablet_index) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:163: +1, including nesting penalty of 0, nesting level increased to 1
if (_opts.rowset_ctx != nullptr) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:168: +1, including nesting penalty of 0, nesting level increased to 1
if (_opts.write_type == DataWriteType::TYPE_DIRECT &&
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:174: +1, including nesting penalty of 0, nesting level increased to 1
if (!InvertedIndexColumnWriter::check_column_valid(column)) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:213: +1, including nesting penalty of 0, nesting level increased to 1
if (column.is_row_store_column()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:218: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(ColumnWriter::create(opts, &column, _file_writer, &writer));
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:218: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(ColumnWriter::create(opts, &column, _file_writer, &writer));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:219: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(writer->init());
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:219: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(writer->init());
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^| @@ -308,13 +305,14 @@ | |||
| // 2.2 build read plan to read by batch | |||
| // 2.3 fill block | |||
| // 3. set columns to data convertor and then write all columns | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data) { | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data, | |||
There was a problem hiding this comment.
warning: function '_append_block_with_partial_content' has cognitive complexity of 57 (threshold 50) [readability-function-cognitive-complexity]
Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:351: +1, including nesting penalty of 0, nesting level increased to 1
if (const vectorized::ColumnWithTypeAndName* delete_sign_column =
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:356: +2, including nesting penalty of 1, nesting level increased to 2
if (delete_sign_col.size() >= data.row_pos + data.num_rows) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:363: +1, including nesting penalty of 0, nesting level increased to 1
DBUG_EXECUTE_IF("VerticalSegmentWriter._append_block_with_partial_content.sleep",
^be/src/util/debug_points.h:35: expanded from macro 'DBUG_EXECUTE_IF'
if (UNLIKELY(config::enable_debug_points)) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:363: +2, including nesting penalty of 1, nesting level increased to 2
DBUG_EXECUTE_IF("VerticalSegmentWriter._append_block_with_partial_content.sleep",
^be/src/util/debug_points.h:37: expanded from macro 'DBUG_EXECUTE_IF'
if (dp) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:367: +1, including nesting penalty of 0, nesting level increased to 1
if (specified_rowsets.size() != _mow_context->rowset_ids.size()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:376: +2, including nesting penalty of 1, nesting level increased to 2
if (_opts.rowset_ctx->partial_update_info->is_strict_mode) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:387: +1, including nesting penalty of 0, nesting level increased to 1
for (size_t block_pos = data.row_pos; block_pos < data.row_pos + data.num_rows; block_pos++) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:398: +2, including nesting penalty of 1, nesting level increased to 2
if (have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:404: +2, including nesting penalty of 1, nesting level increased to 2
if (!_tablet_schema->has_sequence_col() || have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:405: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:405: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:410: +1
(delete_sign_column_data != nullptr && delete_sign_column_data[block_pos] != 0);
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:417: +2, including nesting penalty of 1, nesting level increased to 2
if (st.is<KEY_NOT_FOUND>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:418: +3, including nesting penalty of 2, nesting level increased to 3
if (_opts.rowset_ctx->partial_update_info->is_strict_mode) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:424: +1, nesting level increased to 3
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:425: +4, including nesting penalty of 3, nesting level increased to 4
if (!_opts.rowset_ctx->partial_update_info->can_insert_new_rows_in_partial_update) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:446: +2, including nesting penalty of 1, nesting level increased to 2
if (!st.ok() && !st.is<KEY_ALREADY_EXISTS>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:456: +2, including nesting penalty of 1, nesting level increased to 2
if (have_delete_sign && !_tablet_schema->has_sequence_col()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:459: +1, nesting level increased to 2
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:466: +2, including nesting penalty of 1, nesting level increased to 2
if (st.is<KEY_ALREADY_EXISTS>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:472: +1, nesting level increased to 2
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:479: +1, including nesting penalty of 0, nesting level increased to 1
if (config::enable_merge_on_write_correctness_check) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:486: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(_fill_missing_columns(mutable_full_columns, use_default_or_null_flag,
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:486: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(_fill_missing_columns(mutable_full_columns, use_default_or_null_flag,
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:489: +1, including nesting penalty of 0, nesting level increased to 1
if (_tablet_schema->store_row_column()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:513: +1, including nesting penalty of 0, nesting level increased to 1
if (_tablet_schema->has_sequence_col() && !have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:520: +2, including nesting penalty of 1, nesting level increased to 2
if (_num_rows_written != data.row_pos ||
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:527: +2, including nesting penalty of 1, nesting level increased to 2
for (size_t block_pos = data.row_pos; block_pos < data.row_pos + data.num_rows;
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:531: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:531: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^|
|
||
| Status parse_variant_columns(Block& block, const std::vector<int>& variant_pos, | ||
| const ParseContext& ctx) { | ||
| Status _parse_variant_columns(Block& block, const std::vector<int>& variant_pos, |
There was a problem hiding this comment.
warning: function '_parse_variant_columns' has cognitive complexity of 59 (threshold 50) [readability-function-cognitive-complexity]
Status _parse_variant_columns(Block& block, const std::vector<int>& variant_pos,
^Additional context
be/src/vec/common/schema_util.cpp:481: +1, including nesting penalty of 0, nesting level increased to 1
for (int i = 0; i < variant_pos.size(); ++i) {
^be/src/vec/common/schema_util.cpp:491: nesting level increased to 2
auto encode_rowstore = [&]() {
^be/src/vec/common/schema_util.cpp:492: +3, including nesting penalty of 2, nesting level increased to 3
if (!ctx.record_raw_json_column) {
^be/src/vec/common/schema_util.cpp:496: +3, including nesting penalty of 2, nesting level increased to 3
if (record_raw_string_with_serialization) {
^be/src/vec/common/schema_util.cpp:499: +4, including nesting penalty of 3, nesting level increased to 4
for (size_t i = 0; i < var->rows(); ++i) {
^be/src/vec/common/schema_util.cpp:501: +5, including nesting penalty of 4, nesting level increased to 5
RETURN_IF_ERROR(var->serialize_one_row_to_string(i, &raw_str));
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:501: +6, including nesting penalty of 5, nesting level increased to 6
RETURN_IF_ERROR(var->serialize_one_row_to_string(i, &raw_str));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:505: +1, nesting level increased to 3
} else {
^be/src/vec/common/schema_util.cpp:515: +2, including nesting penalty of 1, nesting level increased to 2
if (!var.is_scalar_variant()) {
^be/src/vec/common/schema_util.cpp:518: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:518: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:523: +2, including nesting penalty of 1, nesting level increased to 2
if (WhichDataType(remove_nullable(var.get_root_type())).is_json()) {
^be/src/vec/common/schema_util.cpp:526: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(cast_column({var.get_root(), var.get_root_type(), ""},
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:528: +4, including nesting penalty of 3, nesting level increased to 4
? make_nullable(std::make_shared<DataTypeString>())
^be/src/vec/common/schema_util.cpp:526: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(cast_column({var.get_root(), var.get_root_type(), ""},
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:531: +3, including nesting penalty of 2, nesting level increased to 3
if (raw_json_column->is_nullable()) {
^be/src/vec/common/schema_util.cpp:535: +1, nesting level increased to 2
} else {
^be/src/vec/common/schema_util.cpp:539: +3, including nesting penalty of 2, nesting level increased to 3
? assert_cast<const ColumnNullable&>(root).get_nested_column_ptr()
^be/src/vec/common/schema_util.cpp:549: +2, including nesting penalty of 1, nesting level increased to 2
if (is_nullable) {
^be/src/vec/common/schema_util.cpp:555: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:614: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:555: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^
|
TeamCity be ut coverage result: |
|
run buildall |
| } | ||
|
|
||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column) { | ||
| Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column, |
There was a problem hiding this comment.
warning: function '_create_column_writer' has cognitive complexity of 51 (threshold 50) [readability-function-cognitive-complexity]
Status VerticalSegmentWriter::_create_column_writer(uint32_t cid, const TabletColumn& column,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:154: +1, including nesting penalty of 0, nesting level increased to 1
if (tablet_index) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:163: +1, including nesting penalty of 0, nesting level increased to 1
if (_opts.rowset_ctx != nullptr) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:168: +1, including nesting penalty of 0, nesting level increased to 1
if (_opts.write_type == DataWriteType::TYPE_DIRECT &&
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:174: +1, including nesting penalty of 0, nesting level increased to 1
if (!InvertedIndexColumnWriter::check_column_valid(column)) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:202: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(STRUCT, "struct")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:203: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(ARRAY, "array")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:204: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(JSONB, "jsonb")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:205: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(AGG_STATE, "agg_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:206: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(MAP, "map")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:207: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(OBJECT, "object")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:208: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(HLL, "hll")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +1, including nesting penalty of 0, nesting level increased to 1
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:192: expanded from macro 'CHECK_FIELD_TYPE'
if (column.type() == FieldType::OLAP_FIELD_TYPE_##TYPE) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:194: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bloom_filter) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:209: +2, including nesting penalty of 1, nesting level increased to 2
CHECK_FIELD_TYPE(QUANTILE_STATE, "quantile_state")
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:197: expanded from macro 'CHECK_FIELD_TYPE'
if (opts.need_bitmap_index) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:213: +1, including nesting penalty of 0, nesting level increased to 1
if (column.is_row_store_column()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:218: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(ColumnWriter::create(opts, &column, _file_writer, &writer));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:218: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(ColumnWriter::create(opts, &column, _file_writer, &writer));
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:219: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(writer->init());
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:219: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(writer->init());
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^| @@ -308,13 +305,14 @@ | |||
| // 2.2 build read plan to read by batch | |||
| // 2.3 fill block | |||
| // 3. set columns to data convertor and then write all columns | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data) { | |||
| Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data, | |||
There was a problem hiding this comment.
warning: function '_append_block_with_partial_content' has cognitive complexity of 57 (threshold 50) [readability-function-cognitive-complexity]
Status VerticalSegmentWriter::_append_block_with_partial_content(RowsInBlock& data,
^Additional context
be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:351: +1, including nesting penalty of 0, nesting level increased to 1
if (const vectorized::ColumnWithTypeAndName* delete_sign_column =
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:356: +2, including nesting penalty of 1, nesting level increased to 2
if (delete_sign_col.size() >= data.row_pos + data.num_rows) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:363: +1, including nesting penalty of 0, nesting level increased to 1
DBUG_EXECUTE_IF("VerticalSegmentWriter._append_block_with_partial_content.sleep",
^be/src/util/debug_points.h:35: expanded from macro 'DBUG_EXECUTE_IF'
if (UNLIKELY(config::enable_debug_points)) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:363: +2, including nesting penalty of 1, nesting level increased to 2
DBUG_EXECUTE_IF("VerticalSegmentWriter._append_block_with_partial_content.sleep",
^be/src/util/debug_points.h:37: expanded from macro 'DBUG_EXECUTE_IF'
if (dp) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:367: +1, including nesting penalty of 0, nesting level increased to 1
if (specified_rowsets.size() != _mow_context->rowset_ids.size()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:376: +2, including nesting penalty of 1, nesting level increased to 2
if (_opts.rowset_ctx->partial_update_info->is_strict_mode) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:387: +1, including nesting penalty of 0, nesting level increased to 1
for (size_t block_pos = data.row_pos; block_pos < data.row_pos + data.num_rows; block_pos++) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:398: +2, including nesting penalty of 1, nesting level increased to 2
if (have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:404: +2, including nesting penalty of 1, nesting level increased to 2
if (!_tablet_schema->has_sequence_col() || have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:405: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:405: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:410: +1
(delete_sign_column_data != nullptr && delete_sign_column_data[block_pos] != 0);
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:417: +2, including nesting penalty of 1, nesting level increased to 2
if (st.is<KEY_NOT_FOUND>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:418: +3, including nesting penalty of 2, nesting level increased to 3
if (_opts.rowset_ctx->partial_update_info->is_strict_mode) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:424: +1, nesting level increased to 3
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:425: +4, including nesting penalty of 3, nesting level increased to 4
if (!_opts.rowset_ctx->partial_update_info->can_insert_new_rows_in_partial_update) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:446: +2, including nesting penalty of 1, nesting level increased to 2
if (!st.ok() && !st.is<KEY_ALREADY_EXISTS>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:456: +2, including nesting penalty of 1, nesting level increased to 2
if (have_delete_sign && !_tablet_schema->has_sequence_col()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:459: +1, nesting level increased to 2
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:466: +2, including nesting penalty of 1, nesting level increased to 2
if (st.is<KEY_ALREADY_EXISTS>()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:472: +1, nesting level increased to 2
} else {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:479: +1, including nesting penalty of 0, nesting level increased to 1
if (config::enable_merge_on_write_correctness_check) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:486: +1, including nesting penalty of 0, nesting level increased to 1
RETURN_IF_ERROR(_fill_missing_columns(mutable_full_columns, use_default_or_null_flag,
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:486: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(_fill_missing_columns(mutable_full_columns, use_default_or_null_flag,
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:489: +1, including nesting penalty of 0, nesting level increased to 1
if (_tablet_schema->store_row_column()) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:513: +1, including nesting penalty of 0, nesting level increased to 1
if (_tablet_schema->has_sequence_col() && !have_input_seq_column) {
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:520: +2, including nesting penalty of 1, nesting level increased to 2
if (_num_rows_written != data.row_pos ||
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:527: +2, including nesting penalty of 1, nesting level increased to 2
for (size_t block_pos = data.row_pos; block_pos < data.row_pos + data.num_rows;
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:531: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/olap/rowset/segment_v2/vertical_segment_writer.cpp:531: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(_primary_key_index_builder->add_item(key));
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^|
|
||
| Status parse_variant_columns(Block& block, const std::vector<int>& variant_pos, | ||
| const ParseContext& ctx) { | ||
| Status _parse_variant_columns(Block& block, const std::vector<int>& variant_pos, |
There was a problem hiding this comment.
warning: function '_parse_variant_columns' has cognitive complexity of 59 (threshold 50) [readability-function-cognitive-complexity]
Status _parse_variant_columns(Block& block, const std::vector<int>& variant_pos,
^Additional context
be/src/vec/common/schema_util.cpp:485: +1, including nesting penalty of 0, nesting level increased to 1
for (int i = 0; i < variant_pos.size(); ++i) {
^be/src/vec/common/schema_util.cpp:495: nesting level increased to 2
auto encode_rowstore = [&]() {
^be/src/vec/common/schema_util.cpp:496: +3, including nesting penalty of 2, nesting level increased to 3
if (!ctx.record_raw_json_column) {
^be/src/vec/common/schema_util.cpp:500: +3, including nesting penalty of 2, nesting level increased to 3
if (record_raw_string_with_serialization) {
^be/src/vec/common/schema_util.cpp:503: +4, including nesting penalty of 3, nesting level increased to 4
for (size_t i = 0; i < var->rows(); ++i) {
^be/src/vec/common/schema_util.cpp:505: +5, including nesting penalty of 4, nesting level increased to 5
RETURN_IF_ERROR(var->serialize_one_row_to_string(i, &raw_str));
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:505: +6, including nesting penalty of 5, nesting level increased to 6
RETURN_IF_ERROR(var->serialize_one_row_to_string(i, &raw_str));
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:509: +1, nesting level increased to 3
} else {
^be/src/vec/common/schema_util.cpp:519: +2, including nesting penalty of 1, nesting level increased to 2
if (!var.is_scalar_variant()) {
^be/src/vec/common/schema_util.cpp:522: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:522: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:527: +2, including nesting penalty of 1, nesting level increased to 2
if (WhichDataType(remove_nullable(var.get_root_type())).is_json()) {
^be/src/vec/common/schema_util.cpp:530: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(cast_column({var.get_root(), var.get_root_type(), ""},
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:532: +4, including nesting penalty of 3, nesting level increased to 4
? make_nullable(std::make_shared<DataTypeString>())
^be/src/vec/common/schema_util.cpp:530: +4, including nesting penalty of 3, nesting level increased to 4
RETURN_IF_ERROR(cast_column({var.get_root(), var.get_root_type(), ""},
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^be/src/vec/common/schema_util.cpp:535: +3, including nesting penalty of 2, nesting level increased to 3
if (raw_json_column->is_nullable()) {
^be/src/vec/common/schema_util.cpp:539: +1, nesting level increased to 2
} else {
^be/src/vec/common/schema_util.cpp:543: +3, including nesting penalty of 2, nesting level increased to 3
? assert_cast<const ColumnNullable&>(root).get_nested_column_ptr()
^be/src/vec/common/schema_util.cpp:553: +2, including nesting penalty of 1, nesting level increased to 2
if (is_nullable) {
^be/src/vec/common/schema_util.cpp:559: +2, including nesting penalty of 1, nesting level increased to 2
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:616: expanded from macro 'RETURN_IF_ERROR'
do { \
^be/src/vec/common/schema_util.cpp:559: +3, including nesting penalty of 2, nesting level increased to 3
RETURN_IF_ERROR(encode_rowstore());
^be/src/common/status.h:618: expanded from macro 'RETURN_IF_ERROR'
if (UNLIKELY(!_status_.ok())) { \
^
|
TeamCity be ut coverage result: |
|
TeamCity be ut coverage result: |
TPC-DS: Total hot run time: 173010 ms |
ClickBench: Total hot run time: 31.61 s |
|
PR approved by at least one committer and no changes requested. |
github-actions
bot
added
approved
|
PR approved by anyone and no changes requested. |
``` Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnObject Check failure stack trace: *** @ 0x562dcebca976 google::LogMessage::SendToLog() @ 0x562dcebc73c0 google::LogMessage::Flush() @ 0x562dcebcb1b9 google::LogMessageFatal::~LogMessageFatal() @ 0x562d9ded39e6 assert_cast<>() @ 0x562d9df1e599 doris::segment_v2::HierarchicalDataReader::process_read<>() @ 0x562d9df1e106 doris::segment_v2::HierarchicalDataReader::next_batch() @ 0x562d9df3373e doris::segment_v2::ColumnIterator::next_batch() .... @ 0x562d9ea56d31 doris::EngineChecksumTask::_compute_checksum() @ 0x562d9ea55cc2 doris::EngineChecksumTask::execute() @ 0x562d9b558355 doris::check_consistency_callback() ``` introduced by apache#34925
) ``` Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnObject Check failure stack trace: *** @ 0x562dcebca976 google::LogMessage::SendToLog() @ 0x562dcebc73c0 google::LogMessage::Flush() @ 0x562dcebcb1b9 google::LogMessageFatal::~LogMessageFatal() @ 0x562d9ded39e6 assert_cast<>() @ 0x562d9df1e599 doris::segment_v2::HierarchicalDataReader::process_read<>() @ 0x562d9df1e106 doris::segment_v2::HierarchicalDataReader::next_batch() @ 0x562d9df3373e doris::segment_v2::ColumnIterator::next_batch() .... @ 0x562d9ea56d31 doris::EngineChecksumTask::_compute_checksum() @ 0x562d9ea55cc2 doris::EngineChecksumTask::execute() @ 0x562d9b558355 doris::check_consistency_callback() ``` introduced by #34925
…34925) 1. Moved variant flatten and flush logic to `_append_block_with_variant_subcolumns` in segment writer to simplify handling partial updates. 2. Ensured rowset schema is refreshed during partial updates for tables with variant types, preventing stale schemas and incorrect data reads. 3. Removed `_output_as_raw_json` as the segment writer now handles variant flatten and flush logic, which was used for schema change, but t flatten and flush already handled in segment writer now. 4. Implemented element_at function in BE for non-scalar variants to enhance functionality.
) ``` Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnObject Check failure stack trace: *** @ 0x562dcebca976 google::LogMessage::SendToLog() @ 0x562dcebc73c0 google::LogMessage::Flush() @ 0x562dcebcb1b9 google::LogMessageFatal::~LogMessageFatal() @ 0x562d9ded39e6 assert_cast<>() @ 0x562d9df1e599 doris::segment_v2::HierarchicalDataReader::process_read<>() @ 0x562d9df1e106 doris::segment_v2::HierarchicalDataReader::next_batch() @ 0x562d9df3373e doris::segment_v2::ColumnIterator::next_batch() .... @ 0x562d9ea56d31 doris::EngineChecksumTask::_compute_checksum() @ 0x562d9ea55cc2 doris::EngineChecksumTask::execute() @ 0x562d9b558355 doris::check_consistency_callback() ``` introduced by #34925
) 1. Variant use serialize_one_row_to_string to string and then parse to jsonb as row store.Since we could not get the original string after #34925 2. Remove redundant code
) 1. Variant use serialize_one_row_to_string to string and then parse to jsonb as row store.Since we could not get the original string after #34925 2. Remove redundant code
…pache#34925) 1. Moved variant flatten and flush logic to `_append_block_with_variant_subcolumns` in segment writer to simplify handling partial updates. 2. Ensured rowset schema is refreshed during partial updates for tables with variant types, preventing stale schemas and incorrect data reads. 3. Removed `_output_as_raw_json` as the segment writer now handles variant flatten and flush logic, which was used for schema change, but t flatten and flush already handled in segment writer now. 4. Implemented element_at function in BE for non-scalar variants to enhance functionality.
…pache#34925) 1. Moved variant flatten and flush logic to `_append_block_with_variant_subcolumns` in segment writer to simplify handling partial updates. 2. Ensured rowset schema is refreshed during partial updates for tables with variant types, preventing stale schemas and incorrect data reads. 3. Removed `_output_as_raw_json` as the segment writer now handles variant flatten and flush logic, which was used for schema change, but t flatten and flush already handled in segment writer now. 4. Implemented element_at function in BE for non-scalar variants to enhance functionality.
…che#36201) ``` Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnObject Check failure stack trace: *** @ 0x562dcebca976 google::LogMessage::SendToLog() @ 0x562dcebc73c0 google::LogMessage::Flush() @ 0x562dcebcb1b9 google::LogMessageFatal::~LogMessageFatal() @ 0x562d9ded39e6 assert_cast<>() @ 0x562d9df1e599 doris::segment_v2::HierarchicalDataReader::process_read<>() @ 0x562d9df1e106 doris::segment_v2::HierarchicalDataReader::next_batch() @ 0x562d9df3373e doris::segment_v2::ColumnIterator::next_batch() .... @ 0x562d9ea56d31 doris::EngineChecksumTask::_compute_checksum() @ 0x562d9ea55cc2 doris::EngineChecksumTask::execute() @ 0x562d9b558355 doris::check_consistency_callback() ``` introduced by apache#34925
Proposed changes
_append_block_with_variant_subcolumnsin segment writer to simplify handling partial updates._output_as_raw_jsonas the segment writer now handles variant flatten and flush logic, which was used for schema change, but t flatten and flush already handled in segment writer now.Issue Number: close #xxx
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...