Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug](compatibility) fix covar_samp function coredump when upgrade #41023

Merged
merged 3 commits into from
Sep 20, 2024

Conversation

zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Sep 19, 2024

Proposed changes

in branch-2.1, the function covar_samp mode is PropagateNullable
eg: select covar_samp(non_nullable(x), y) from test_covar_samp;
x, y has nullable, and return is nullable
and now the column of non_nullable(x), y is nullable should check is nullable alone.

the fixed in branch-21
#39943

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@zhangstar333
Copy link
Contributor Author

run buildall

dataroaring
dataroaring previously approved these changes Sep 19, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 19, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 41728 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3c78704dfbc0821e596637a55fa110cc6293b765, data reload: false

------ Round 1 ----------------------------------
q1	18163	8173	7470	7470
q2	2535	186	177	177
q3	11308	1153	1152	1152
q4	10667	752	773	752
q5	8055	3107	3114	3107
q6	234	151	152	151
q7	1024	636	628	628
q8	9743	2018	2027	2018
q9	6833	6354	6363	6354
q10	7007	2275	2273	2273
q11	440	242	243	242
q12	401	214	211	211
q13	17754	2972	3004	2972
q14	250	212	209	209
q15	581	513	518	513
q16	679	632	593	593
q17	964	826	802	802
q18	7269	6806	6705	6705
q19	1398	979	1103	979
q20	594	290	283	283
q21	3937	3140	3209	3140
q22	1079	997	1035	997
Total cold run time: 110915 ms
Total hot run time: 41728 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7229	7283	7230	7230
q2	326	230	231	230
q3	2924	2756	2797	2756
q4	1921	1740	1711	1711
q5	5357	5382	5397	5382
q6	227	145	144	144
q7	2057	1699	1721	1699
q8	3157	3314	3328	3314
q9	8389	8381	8359	8359
q10	3367	3354	3348	3348
q11	564	475	466	466
q12	767	593	593	593
q13	5281	2997	3018	2997
q14	282	256	254	254
q15	552	514	506	506
q16	698	682	649	649
q17	1747	1549	1524	1524
q18	7695	7354	7189	7189
q19	1665	1576	1384	1384
q20	2043	1804	1814	1804
q21	5353	5228	5238	5228
q22	1133	1039	1021	1021
Total cold run time: 62734 ms
Total hot run time: 57788 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.32% (9590/25698)
Line Coverage: 28.71% (79269/276128)
Region Coverage: 28.17% (41038/145683)
Branch Coverage: 24.79% (20904/84330)
Coverage Report: http://coverage.selectdb-in.cc/coverage/3c78704dfbc0821e596637a55fa110cc6293b765_3c78704dfbc0821e596637a55fa110cc6293b765/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 190747 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3c78704dfbc0821e596637a55fa110cc6293b765, data reload: false

query1	980	369	365	365
query2	6525	2093	2091	2091
query3	6699	212	214	212
query4	34349	23462	23441	23441
query5	4363	480	479	479
query6	262	160	178	160
query7	4620	295	299	295
query8	277	254	230	230
query9	9564	2692	2711	2692
query10	473	297	294	294
query11	18325	15255	15200	15200
query12	158	99	98	98
query13	1623	415	388	388
query14	11067	7273	7426	7273
query15	298	173	178	173
query16	8017	467	455	455
query17	1650	580	570	570
query18	2103	303	313	303
query19	371	151	147	147
query20	115	103	115	103
query21	211	103	104	103
query22	4438	4156	4200	4156
query23	34934	33998	34212	33998
query24	11193	2978	2894	2894
query25	658	403	399	399
query26	1372	164	166	164
query27	2819	293	293	293
query28	8251	2472	2469	2469
query29	916	425	433	425
query30	321	165	158	158
query31	1068	802	827	802
query32	100	57	62	57
query33	766	311	299	299
query34	927	494	488	488
query35	860	739	730	730
query36	1083	932	948	932
query37	163	90	88	88
query38	4022	3884	3892	3884
query39	1548	1416	1415	1415
query40	289	100	99	99
query41	52	49	48	48
query42	118	99	95	95
query43	525	491	502	491
query44	1254	833	799	799
query45	196	169	172	169
query46	1132	782	786	782
query47	1904	1787	1835	1787
query48	479	374	377	374
query49	1152	424	417	417
query50	829	414	411	411
query51	7100	6919	6907	6907
query52	96	90	93	90
query53	255	184	187	184
query54	1195	479	492	479
query55	89	76	80	76
query56	302	265	263	263
query57	1221	1092	1077	1077
query58	257	252	250	250
query59	3323	3100	2936	2936
query60	308	282	266	266
query61	144	102	105	102
query62	849	657	686	657
query63	227	196	187	187
query64	5255	643	631	631
query65	3243	3164	3192	3164
query66	1411	306	307	306
query67	15823	15731	15407	15407
query68	4814	584	578	578
query69	464	292	297	292
query70	1210	1133	1101	1101
query71	340	274	272	272
query72	6341	3757	3938	3757
query73	765	322	336	322
query74	9783	8875	8972	8875
query75	3381	2609	2647	2609
query76	3218	992	861	861
query77	436	304	299	299
query78	9824	9187	9140	9140
query79	1293	558	560	558
query80	1086	457	449	449
query81	530	242	243	242
query82	923	146	142	142
query83	204	139	143	139
query84	241	76	81	76
query85	1286	295	289	289
query86	352	296	295	295
query87	4476	4334	4338	4334
query88	3086	2357	2357	2357
query89	387	282	282	282
query90	1841	188	193	188
query91	180	141	144	141
query92	65	50	47	47
query93	1062	549	545	545
query94	874	304	286	286
query95	360	258	266	258
query96	614	282	277	277
query97	3281	3066	3170	3066
query98	220	199	196	196
query99	1538	1302	1297	1297
Total cold run time: 300992 ms
Total hot run time: 190747 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.45 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3c78704dfbc0821e596637a55fa110cc6293b765, data reload: false

query1	0.05	0.04	0.04
query2	0.06	0.03	0.03
query3	0.23	0.06	0.07
query4	1.65	0.10	0.10
query5	0.52	0.49	0.51
query6	1.13	0.72	0.72
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.49
query10	0.54	0.57	0.54
query11	0.14	0.10	0.10
query12	0.13	0.10	0.11
query13	0.61	0.59	0.59
query14	2.95	2.95	3.02
query15	0.88	0.81	0.82
query16	0.39	0.39	0.38
query17	1.08	1.02	1.05
query18	0.22	0.21	0.21
query19	1.93	1.86	2.00
query20	0.00	0.01	0.01
query21	15.34	0.59	0.59
query22	2.87	2.24	1.72
query23	17.34	0.85	0.89
query24	2.82	1.29	0.80
query25	0.19	0.12	0.04
query26	0.53	0.14	0.13
query27	0.04	0.04	0.03
query28	10.91	1.09	1.07
query29	12.54	3.19	3.22
query30	0.25	0.06	0.07
query31	2.87	0.38	0.38
query32	3.28	0.48	0.46
query33	3.00	3.05	3.03
query34	16.89	4.38	4.37
query35	4.46	4.42	4.44
query36	0.66	0.48	0.49
query37	0.08	0.06	0.06
query38	0.05	0.03	0.04
query39	0.03	0.02	0.02
query40	0.16	0.12	0.12
query41	0.07	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.02
Total cold run time: 107.57 s
Total hot run time: 32.45 s

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 20, 2024
@zhangstar333
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.32% (9591/25698)
Line Coverage: 28.72% (79303/276132)
Region Coverage: 28.17% (41035/145690)
Branch Coverage: 24.79% (20910/84336)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4c83283ad9bc4e66b21c6d141b2228b8a755df9a_4c83283ad9bc4e66b21c6d141b2228b8a755df9a/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 41269 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4c83283ad9bc4e66b21c6d141b2228b8a755df9a, data reload: false

------ Round 1 ----------------------------------
q1	17571	7880	7210	7210
q2	2056	161	164	161
q3	10584	1070	1188	1070
q4	10462	740	842	740
q5	7754	3036	3047	3036
q6	232	149	146	146
q7	987	616	602	602
q8	9444	2032	1993	1993
q9	6825	6410	6409	6409
q10	7062	2256	2282	2256
q11	444	249	274	249
q12	406	212	213	212
q13	17791	3007	2961	2961
q14	257	223	216	216
q15	572	558	524	524
q16	714	614	634	614
q17	972	798	807	798
q18	7303	6706	6758	6706
q19	1417	1026	948	948
q20	576	291	291	291
q21	3988	3211	3123	3123
q22	1121	1004	1017	1004
Total cold run time: 108538 ms
Total hot run time: 41269 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7245	7241	7194	7194
q2	329	230	225	225
q3	3033	2942	2970	2942
q4	2011	1870	1875	1870
q5	5567	5568	5571	5568
q6	238	142	146	142
q7	2203	1853	1752	1752
q8	3299	3446	3423	3423
q9	8720	8807	8734	8734
q10	3566	3455	3425	3425
q11	566	490	500	490
q12	824	651	659	651
q13	9616	3174	3170	3170
q14	306	278	284	278
q15	574	524	513	513
q16	712	679	678	678
q17	1809	1588	1572	1572
q18	8201	7829	7864	7829
q19	1724	1586	1569	1569
q20	2180	1855	1866	1855
q21	5547	5361	5397	5361
q22	1149	1040	1049	1040
Total cold run time: 69419 ms
Total hot run time: 60281 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 195496 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4c83283ad9bc4e66b21c6d141b2228b8a755df9a, data reload: false

query1	1265	856	861	856
query2	6216	2164	2129	2129
query3	10812	3947	4029	3947
query4	63746	26923	23563	23563
query5	5064	477	465	465
query6	410	169	184	169
query7	5449	308	291	291
query8	302	213	201	201
query9	8559	2674	2675	2674
query10	460	281	295	281
query11	17602	15309	15828	15309
query12	168	99	102	99
query13	1452	423	425	423
query14	10762	7509	7400	7400
query15	206	174	177	174
query16	6856	477	474	474
query17	1234	610	592	592
query18	1528	301	320	301
query19	206	158	153	153
query20	135	114	116	114
query21	206	100	103	100
query22	4711	4595	4719	4595
query23	34662	33841	33859	33841
query24	5991	2875	2875	2875
query25	513	411	409	409
query26	614	158	156	156
query27	1595	283	298	283
query28	4131	2469	2427	2427
query29	675	432	422	422
query30	231	156	149	149
query31	971	766	807	766
query32	72	56	54	54
query33	420	300	300	300
query34	904	495	491	491
query35	832	726	716	716
query36	1055	917	924	917
query37	152	83	83	83
query38	4065	4029	3955	3955
query39	1474	1425	1424	1424
query40	210	96	95	95
query41	51	48	49	48
query42	113	95	94	94
query43	544	484	498	484
query44	1135	806	778	778
query45	189	163	163	163
query46	1129	786	735	735
query47	1934	1830	1862	1830
query48	468	366	364	364
query49	701	399	397	397
query50	853	401	402	401
query51	7067	6876	6942	6876
query52	96	86	91	86
query53	253	179	183	179
query54	587	453	463	453
query55	75	74	72	72
query56	265	250	241	241
query57	1254	1115	1084	1084
query58	225	242	237	237
query59	3401	3003	2968	2968
query60	309	255	259	255
query61	101	102	100	100
query62	752	663	660	660
query63	214	184	177	177
query64	1292	647	617	617
query65	3249	3166	3163	3163
query66	672	282	298	282
query67	15736	15444	15585	15444
query68	4379	584	567	567
query69	511	375	319	319
query70	1171	1048	1144	1048
query71	409	267	257	257
query72	6438	4056	3953	3953
query73	783	330	331	330
query74	9478	8969	9112	8969
query75	3362	2608	2707	2608
query76	1737	873	968	873
query77	563	307	295	295
query78	9906	9176	9178	9176
query79	1569	529	522	522
query80	1673	431	432	431
query81	569	237	240	237
query82	1247	143	137	137
query83	232	135	138	135
query84	285	80	79	79
query85	842	280	267	267
query86	330	297	297	297
query87	4462	4223	4251	4223
query88	3912	2333	2341	2333
query89	407	282	287	282
query90	1933	184	182	182
query91	178	142	146	142
query92	56	46	47	46
query93	1712	551	540	540
query94	707	298	285	285
query95	341	251	255	251
query96	609	284	276	276
query97	3186	3129	3102	3102
query98	212	211	193	193
query99	1801	1291	1280	1280
Total cold run time: 313581 ms
Total hot run time: 195496 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.69 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4c83283ad9bc4e66b21c6d141b2228b8a755df9a, data reload: false

query1	0.05	0.05	0.04
query2	0.06	0.03	0.03
query3	0.22	0.06	0.06
query4	1.65	0.10	0.10
query5	0.52	0.53	0.51
query6	1.13	0.73	0.74
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.56	0.50	0.51
query10	0.57	0.56	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.10
query13	0.61	0.59	0.58
query14	2.98	2.97	2.95
query15	0.90	0.82	0.82
query16	0.38	0.39	0.38
query17	1.01	1.01	1.00
query18	0.21	0.19	0.19
query19	1.90	1.83	1.95
query20	0.01	0.00	0.01
query21	15.36	0.58	0.60
query22	2.78	2.70	2.28
query23	17.48	0.86	0.75
query24	3.04	0.98	0.69
query25	0.29	0.12	0.07
query26	0.38	0.14	0.14
query27	0.04	0.05	0.03
query28	11.43	1.13	1.08
query29	12.63	3.25	3.22
query30	0.25	0.06	0.06
query31	2.86	0.38	0.38
query32	3.31	0.46	0.45
query33	2.98	3.01	3.04
query34	16.92	4.35	4.41
query35	4.41	4.45	4.32
query36	0.67	0.49	0.49
query37	0.08	0.05	0.06
query38	0.04	0.04	0.03
query39	0.03	0.02	0.02
query40	0.15	0.12	0.12
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 108.39 s
Total hot run time: 32.69 s

Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 20, 2024
@dataroaring dataroaring merged commit 73daca7 into apache:master Sep 20, 2024
25 of 29 checks passed
dataroaring pushed a commit that referenced this pull request Sep 20, 2024
…41023)

## Proposed changes

in branch-2.1, the function covar_samp mode is PropagateNullable
eg: select covar_samp(non_nullable(x), y) from test_covar_samp;
x, y has nullable, and return is nullable
and now the column of non_nullable(x), y is nullable should check is
nullable alone.

the fixed in branch-21 
#39943

<!--Describe your changes.-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.2-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants