[fix](variant) update least common type in ColumnObject::pop_back #48935

csun5285 · 2025-03-11T12:51:40Z

What problem does this PR solve?

Problem Summary:

In a subcolumn, each time a different type of column is inserted, the least_common_type is updated.
Similarly, each time a column is popped back, the least_common_type needs to be updated as well.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-03-11T12:51:45Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

csun5285 · 2025-03-11T12:51:48Z

run buildall

doris-robot · 2025-03-11T13:15:15Z

TPC-H: Total hot run time: 32563 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit de750486b0bcfe3d511cf57b852ed2d628f3e261, data reload: false

------ Round 1 ----------------------------------
q1	17605	5271	5070	5070
q2	2040	291	160	160
q3	10417	1302	717	717
q4	10225	1011	530	530
q5	7514	2360	2372	2360
q6	188	162	135	135
q7	923	752	611	611
q8	9312	1293	1069	1069
q9	5148	4833	4622	4622
q10	6835	2317	1877	1877
q11	476	288	269	269
q12	353	364	228	228
q13	17791	3739	3132	3132
q14	225	222	215	215
q15	547	492	483	483
q16	638	627	599	599
q17	582	846	423	423
q18	6784	6511	6337	6337
q19	1212	970	573	573
q20	321	345	200	200
q21	2806	2134	1963	1963
q22	1000	1014	990	990
Total cold run time: 102942 ms
Total hot run time: 32563 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5181	5152	5160	5152
q2	238	338	246	246
q3	2164	2653	2238	2238
q4	1420	1800	1388	1388
q5	4219	4142	4144	4142
q6	207	166	123	123
q7	1862	1976	1773	1773
q8	2628	2779	2617	2617
q9	7230	7113	7099	7099
q10	2999	3187	2728	2728
q11	561	511	496	496
q12	683	787	625	625
q13	3516	3913	3292	3292
q14	290	308	280	280
q15	538	503	496	496
q16	643	703	625	625
q17	1134	1648	1338	1338
q18	7756	7530	7449	7449
q19	815	828	923	828
q20	1985	2047	1890	1890
q21	5444	4925	4851	4851
q22	1041	1050	998	998
Total cold run time: 52554 ms
Total hot run time: 50674 ms

doris-robot · 2025-03-11T13:26:45Z

TPC-DS: Total hot run time: 185276 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit de750486b0bcfe3d511cf57b852ed2d628f3e261, data reload: false

query1	993	386	380	380
query2	6519	1910	1892	1892
query3	6788	219	215	215
query4	26616	23779	22894	22894
query5	4287	649	478	478
query6	289	201	183	183
query7	4600	518	290	290
query8	289	235	222	222
query9	8612	2647	2638	2638
query10	475	309	250	250
query11	15552	15110	14911	14911
query12	154	115	105	105
query13	1647	531	400	400
query14	9117	6747	6484	6484
query15	193	186	175	175
query16	7117	634	481	481
query17	937	721	570	570
query18	1955	415	305	305
query19	190	191	173	173
query20	122	117	122	117
query21	208	122	111	111
query22	4167	4190	3947	3947
query23	33887	32971	33029	32971
query24	7669	2394	2372	2372
query25	548	443	393	393
query26	1238	270	161	161
query27	2119	497	331	331
query28	3934	2428	2427	2427
query29	796	547	428	428
query30	285	211	195	195
query31	934	883	774	774
query32	84	63	64	63
query33	556	369	298	298
query34	803	840	521	521
query35	795	805	741	741
query36	959	980	907	907
query37	130	106	78	78
query38	4217	4112	3985	3985
query39	1460	1397	1415	1397
query40	212	128	115	115
query41	61	56	58	56
query42	118	105	108	105
query43	490	508	476	476
query44	1320	797	802	797
query45	178	170	172	170
query46	851	1034	643	643
query47	1733	1792	1715	1715
query48	398	423	315	315
query49	809	525	440	440
query50	726	748	416	416
query51	4204	4156	4119	4119
query52	110	112	102	102
query53	243	269	202	202
query54	506	513	429	429
query55	91	83	85	83
query56	295	282	265	265
query57	1126	1116	1067	1067
query58	313	237	238	237
query59	2665	2887	2742	2742
query60	293	291	257	257
query61	120	118	123	118
query62	805	729	655	655
query63	231	197	202	197
query64	4546	986	662	662
query65	4451	4330	4322	4322
query66	1139	411	299	299
query67	15575	15632	15378	15378
query68	8356	894	511	511
query69	468	310	259	259
query70	1231	1124	1106	1106
query71	462	300	263	263
query72	5392	3628	3911	3628
query73	800	741	362	362
query74	8889	9241	8893	8893
query75	3729	3184	2673	2673
query76	3625	1182	748	748
query77	815	373	286	286
query78	10069	9992	9274	9274
query79	2879	836	585	585
query80	694	516	448	448
query81	465	270	223	223
query82	687	129	98	98
query83	209	167	154	154
query84	283	95	79	79
query85	768	351	305	305
query86	341	298	291	291
query87	4415	4542	4378	4378
query88	3321	2267	2244	2244
query89	428	314	293	293
query90	1959	209	207	207
query91	139	139	109	109
query92	72	59	62	59
query93	1251	1066	586	586
query94	676	418	301	301
query95	351	278	261	261
query96	480	569	277	277
query97	3353	3380	3254	3254
query98	231	212	206	206
query99	1460	1393	1270	1270
Total cold run time: 273148 ms
Total hot run time: 185276 ms

doris-robot · 2025-03-11T13:32:04Z

ClickBench: Total hot run time: 30.92 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit de750486b0bcfe3d511cf57b852ed2d628f3e261, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.60	0.11	0.11
query5	0.56	0.57	0.55
query6	1.22	0.72	0.73
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.53	0.53
query10	0.59	0.60	0.58
query11	0.16	0.11	0.10
query12	0.15	0.12	0.11
query13	0.61	0.60	0.59
query14	2.65	2.80	2.68
query15	0.94	0.83	0.85
query16	0.39	0.39	0.38
query17	1.05	1.06	1.05
query18	0.21	0.20	0.20
query19	1.87	1.90	1.98
query20	0.01	0.01	0.01
query21	15.35	0.87	0.53
query22	0.74	1.27	0.80
query23	14.77	1.36	0.67
query24	6.95	2.12	0.66
query25	0.49	0.33	0.13
query26	0.53	0.16	0.13
query27	0.05	0.05	0.05
query28	9.56	0.87	0.42
query29	12.54	4.02	3.35
query30	0.25	0.09	0.07
query31	2.82	0.57	0.39
query32	3.44	0.55	0.46
query33	3.02	3.03	3.06
query34	15.77	5.08	4.47
query35	4.55	4.50	4.46
query36	0.66	0.50	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.02
query40	0.16	0.13	0.12
query41	0.09	0.02	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 104.99 s
Total hot run time: 30.92 s

hello-stephen · 2025-03-11T14:36:21Z

BE UT Coverage Report

Increment line coverage 66.67% (2/3) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	47.03% (12578/26744)
Line Coverage	36.68% (107400/292827)
Region Coverage	35.75% (54947/153685)
Branch Coverage	31.15% (27694/88896)

be/test/vec/columns/column_object_test.cpp

eldenmoon · 2025-03-11T14:46:32Z

add more comment to this PR to describe the details of the problem

csun5285 · 2025-03-12T02:14:43Z

run buildall

eldenmoon

LGTM

github-actions · 2025-03-12T02:25:38Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-03-12T02:25:41Z

PR approved by anyone and no changes requested.

doris-robot · 2025-03-12T03:00:47Z

TPC-H: Total hot run time: 32476 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b9716ff7fab9d070e8aaaf7a79013cda5cb6fce3, data reload: false

------ Round 1 ----------------------------------
q1	17625	5241	5095	5095
q2	2041	286	161	161
q3	10443	1270	736	736
q4	10276	1017	531	531
q5	8204	2364	2355	2355
q6	191	160	135	135
q7	913	750	608	608
q8	9320	1285	1097	1097
q9	4956	4662	4939	4662
q10	6962	2303	1909	1909
q11	459	278	257	257
q12	353	362	223	223
q13	17763	3645	3124	3124
q14	226	232	208	208
q15	530	478	484	478
q16	604	617	593	593
q17	581	916	335	335
q18	6755	6349	6392	6349
q19	2320	954	567	567
q20	321	314	188	188
q21	2864	2121	1922	1922
q22	1055	993	943	943
Total cold run time: 104762 ms
Total hot run time: 32476 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5348	5129	5181	5129
q2	250	330	227	227
q3	2149	2641	2289	2289
q4	1455	1837	1349	1349
q5	4200	4170	4140	4140
q6	205	166	132	132
q7	1928	1905	1764	1764
q8	2652	2604	2577	2577
q9	7251	7132	7175	7132
q10	3013	3188	2771	2771
q11	595	511	491	491
q12	704	788	572	572
q13	3568	3896	3259	3259
q14	303	308	273	273
q15	516	480	481	480
q16	638	672	661	661
q17	1156	1592	1370	1370
q18	7715	7564	7393	7393
q19	834	816	867	816
q20	1992	2061	1890	1890
q21	5538	5082	4784	4784
q22	1114	1085	1012	1012
Total cold run time: 53124 ms
Total hot run time: 50511 ms

doris-robot · 2025-03-12T03:12:38Z

TPC-DS: Total hot run time: 192364 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b9716ff7fab9d070e8aaaf7a79013cda5cb6fce3, data reload: false

query1	1398	988	985	985
query2	6209	1982	1983	1982
query3	11010	4447	4355	4355
query4	26163	23287	23421	23287
query5	4403	655	472	472
query6	302	189	182	182
query7	3971	499	292	292
query8	288	228	225	225
query9	8451	2623	2613	2613
query10	481	291	257	257
query11	15476	15046	14871	14871
query12	163	116	107	107
query13	1568	512	387	387
query14	8859	6795	6857	6795
query15	200	191	174	174
query16	7657	630	470	470
query17	1235	761	592	592
query18	2017	420	317	317
query19	202	194	161	161
query20	127	121	124	121
query21	209	129	112	112
query22	4395	4614	4558	4558
query23	34123	33325	33697	33325
query24	7838	2432	2458	2432
query25	503	465	398	398
query26	1198	272	155	155
query27	2201	511	332	332
query28	4301	2452	2422	2422
query29	727	574	427	427
query30	286	221	187	187
query31	923	864	774	774
query32	78	62	60	60
query33	552	375	331	331
query34	789	852	508	508
query35	820	875	752	752
query36	970	1017	979	979
query37	126	98	78	78
query38	4164	4272	4320	4272
query39	1445	1420	1482	1420
query40	214	119	109	109
query41	61	48	50	48
query42	122	107	108	107
query43	529	545	512	512
query44	1357	803	799	799
query45	177	173	173	173
query46	861	1019	653	653
query47	1838	1883	1765	1765
query48	379	430	299	299
query49	771	510	427	427
query50	691	773	421	421
query51	4250	4336	4317	4317
query52	107	102	98	98
query53	242	262	185	185
query54	492	505	420	420
query55	85	80	79	79
query56	278	278	261	261
query57	1142	1184	1088	1088
query58	258	250	240	240
query59	2798	2956	2825	2825
query60	289	274	263	263
query61	120	121	121	121
query62	813	778	703	703
query63	228	190	189	189
query64	4147	1061	691	691
query65	4621	4461	4463	4461
query66	1043	389	302	302
query67	16048	15419	15122	15122
query68	8464	874	499	499
query69	471	294	262	262
query70	1211	1086	1144	1086
query71	474	299	258	258
query72	5778	3553	4039	3553
query73	747	723	346	346
query74	9003	9143	8861	8861
query75	3853	3180	2661	2661
query76	3610	1179	758	758
query77	783	365	271	271
query78	10005	10129	9322	9322
query79	2052	862	574	574
query80	663	516	488	488
query81	484	261	217	217
query82	226	126	89	89
query83	169	167	152	152
query84	243	91	66	66
query85	739	352	315	315
query86	365	312	285	285
query87	4560	4429	4446	4429
query88	3606	2233	2230	2230
query89	377	315	285	285
query90	1887	202	204	202
query91	139	137	110	110
query92	78	56	55	55
query93	2323	1046	562	562
query94	606	406	311	311
query95	351	264	261	261
query96	488	563	270	270
query97	3237	3402	3277	3277
query98	224	197	201	197
query99	1321	1399	1267	1267
Total cold run time: 278326 ms
Total hot run time: 192364 ms

doris-robot · 2025-03-12T03:17:55Z

ClickBench: Total hot run time: 30.67 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b9716ff7fab9d070e8aaaf7a79013cda5cb6fce3, data reload: false

query1	0.03	0.03	0.04
query2	0.06	0.04	0.03
query3	0.24	0.08	0.06
query4	1.61	0.10	0.11
query5	0.57	0.55	0.54
query6	1.18	0.71	0.71
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.50
query10	0.55	0.61	0.57
query11	0.16	0.11	0.11
query12	0.14	0.10	0.11
query13	0.61	0.60	0.60
query14	2.68	2.80	2.67
query15	0.92	0.83	0.83
query16	0.39	0.38	0.38
query17	1.04	1.04	1.00
query18	0.21	0.20	0.20
query19	1.90	1.81	1.98
query20	0.01	0.01	0.01
query21	15.36	0.93	0.53
query22	0.76	1.06	0.63
query23	15.06	1.36	0.63
query24	7.15	1.65	0.85
query25	0.52	0.20	0.12
query26	0.57	0.16	0.13
query27	0.05	0.05	0.04
query28	9.53	0.90	0.41
query29	12.55	3.93	3.26
query30	0.26	0.09	0.07
query31	2.81	0.59	0.38
query32	3.23	0.54	0.46
query33	2.98	3.13	3.10
query34	15.82	5.07	4.49
query35	4.56	4.49	4.51
query36	0.67	0.49	0.48
query37	0.09	0.06	0.07
query38	0.05	0.04	0.03
query39	0.02	0.02	0.03
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 105.29 s
Total hot run time: 30.67 s

hello-stephen · 2025-03-12T03:56:05Z

BE UT Coverage Report

Increment line coverage 66.67% (2/3) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	47.05% (12583/26744)
Line Coverage	36.70% (107479/292831)
Region Coverage	35.79% (54999/153685)
Branch Coverage	31.21% (27740/88894)

amorynan

LGTM

…8935) 1. In a subcolumn, each time a different type of column is inserted, the `least_common_type` is updated. 2. Similarly, each time a column is popped back, the `least_common_type` needs to be updated as well.

…pop_back #48935 (#48979) Cherry-picked from #48935 Co-authored-by: Sun Chenyang <[email protected]>

eldenmoon added dev/3.0.x dev/2.1.x usercase Important user case type label labels Mar 11, 2025

eldenmoon reviewed Mar 11, 2025

View reviewed changes

be/test/vec/columns/column_object_test.cpp Show resolved Hide resolved

csun5285 added 2 commits March 12, 2025 10:14

[fix](variant) update least common type

f560a37

add more ut cases

b9716ff

csun5285 force-pushed the fix_pop_back_master branch from 6041c2a to b9716ff Compare March 12, 2025 02:14

eldenmoon approved these changes Mar 12, 2025

View reviewed changes

github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Mar 12, 2025

eldenmoon changed the title ~~[fix](variant) update least common type~~ [fix](variant) update least common type in ColumnObject::pop_back Mar 12, 2025

amorynan approved these changes Mar 12, 2025

View reviewed changes

eldenmoon merged commit b982b14 into apache:master Mar 12, 2025
27 of 30 checks passed

github-actions bot mentioned this pull request Mar 12, 2025

branch-3.0: [fix](variant) update least common type in ColumnObject::pop_back #48935 #48978

Open

github-actions bot mentioned this pull request Mar 12, 2025

branch-2.1: [fix](variant) update least common type in ColumnObject::pop_back #48935 #48979

Merged

eldenmoon added the p0_b label Mar 13, 2025

yiguolei pushed a commit that referenced this pull request Mar 13, 2025

branch-2.1: [fix](variant) update least common type in ColumnObject::…

ed2e1ac

…pop_back #48935 (#48979) Cherry-picked from #48935 Co-authored-by: Sun Chenyang <[email protected]>

yiguolei added dev/2.1.9-merged and removed dev/2.1.x labels Mar 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](variant) update least common type in ColumnObject::pop_back #48935

[fix](variant) update least common type in ColumnObject::pop_back #48935

csun5285 commented Mar 11, 2025 •

edited

Loading

hello-stephen commented Mar 11, 2025

csun5285 commented Mar 11, 2025

doris-robot commented Mar 11, 2025

doris-robot commented Mar 11, 2025

doris-robot commented Mar 11, 2025

hello-stephen commented Mar 11, 2025

eldenmoon commented Mar 11, 2025

csun5285 commented Mar 12, 2025

eldenmoon left a comment

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

doris-robot commented Mar 12, 2025

doris-robot commented Mar 12, 2025

doris-robot commented Mar 12, 2025

hello-stephen commented Mar 12, 2025

amorynan left a comment

[fix](variant) update least common type in ColumnObject::pop_back #48935

[fix](variant) update least common type in ColumnObject::pop_back #48935

Conversation

csun5285 commented Mar 11, 2025 • edited Loading

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

hello-stephen commented Mar 11, 2025

csun5285 commented Mar 11, 2025

doris-robot commented Mar 11, 2025

doris-robot commented Mar 11, 2025

doris-robot commented Mar 11, 2025

hello-stephen commented Mar 11, 2025

BE UT Coverage Report

eldenmoon commented Mar 11, 2025

csun5285 commented Mar 12, 2025

eldenmoon left a comment

Choose a reason for hiding this comment

github-actions bot commented Mar 12, 2025

github-actions bot commented Mar 12, 2025

doris-robot commented Mar 12, 2025

doris-robot commented Mar 12, 2025

doris-robot commented Mar 12, 2025

hello-stephen commented Mar 12, 2025

BE UT Coverage Report

amorynan left a comment

Choose a reason for hiding this comment

csun5285 commented Mar 11, 2025 •

edited

Loading