Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug](materialized-view) fix analyze where clause failed on mv #39061 #40501

Merged

Conversation

xiaokang
Copy link
Contributor

@xiaokang xiaokang commented Sep 7, 2024

cherry pick from #39061

…e#39061)

## Proposed changes
fix analyze where clause failed on mv
do not analyze slot after replaceSlot to avoid duplicate columns in desc
@xiaokang
Copy link
Contributor Author

xiaokang commented Sep 7, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Sep 7, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 49526 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1a46f621f371727af73e59461fbc177ad1cb595a, data reload: false

------ Round 1 ----------------------------------
q1	18326	4442	4397	4397
q2	2066	156	139	139
q3	10321	1941	1919	1919
q4	10865	1277	1327	1277
q5	8513	3874	3891	3874
q6	238	122	123	122
q7	2072	1646	1576	1576
q8	9428	2747	2729	2729
q9	10027	9961	10100	9961
q10	8936	3707	3710	3707
q11	433	265	261	261
q12	483	310	311	310
q13	22066	4014	4090	4014
q14	353	342	339	339
q15	535	481	485	481
q16	790	470	461	461
q17	1141	970	946	946
q18	7488	6815	6722	6722
q19	1674	1559	1511	1511
q20	540	323	308	308
q21	4411	4088	4082	4082
q22	486	390	409	390
Total cold run time: 121192 ms
Total hot run time: 49526 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4298	4293	4293
q2	322	226	214	214
q3	4140	4157	4167	4157
q4	2734	2744	2723	2723
q5	7173	6998	7054	6998
q6	232	115	120	115
q7	3237	2847	2930	2847
q8	4315	4418	4454	4418
q9	13733	13618	13558	13558
q10	4207	4225	4253	4225
q11	742	691	667	667
q12	1034	844	858	844
q13	6711	3708	3735	3708
q14	467	439	412	412
q15	493	471	452	452
q16	635	581	578	578
q17	3746	3812	3769	3769
q18	8678	8752	8720	8720
q19	1718	1684	1675	1675
q20	2367	2147	2104	2104
q21	8387	8366	8432	8366
q22	1063	961	941	941
Total cold run time: 80468 ms
Total hot run time: 75784 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211489 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1a46f621f371727af73e59461fbc177ad1cb595a, data reload: false

query1	920	394	431	394
query2	6534	2218	2289	2218
query3	6926	206	208	206
query4	23662	21464	21719	21464
query5	19730	6476	6558	6476
query6	278	215	222	215
query7	4323	292	322	292
query8	239	282	229	229
query9	3041	2609	2551	2551
query10	459	302	306	302
query11	15681	14929	15210	14929
query12	121	74	74	74
query13	1038	447	436	436
query14	17785	13573	13408	13408
query15	364	215	221	215
query16	6475	273	260	260
query17	1756	982	888	888
query18	888	323	316	316
query19	213	153	147	147
query20	75	73	79	73
query21	193	112	94	94
query22	5227	4972	4948	4948
query23	34056	33318	33529	33318
query24	6977	6263	6237	6237
query25	527	426	437	426
query26	1122	159	159	159
query27	2352	292	288	288
query28	6214	2281	2231	2231
query29	2903	2642	2766	2642
query30	245	168	169	168
query31	954	697	768	697
query32	70	59	59	59
query33	432	256	262	256
query34	875	464	491	464
query35	1107	866	931	866
query36	1287	1069	1108	1069
query37	89	58	60	58
query38	3027	2930	2892	2892
query39	1386	1312	1341	1312
query40	307	95	96	95
query41	38	37	36	36
query42	84	86	85	85
query43	675	581	541	541
query44	1178	713	710	710
query45	247	225	232	225
query46	1233	969	975	969
query47	1829	1956	1881	1881
query48	501	409	413	409
query49	640	367	361	361
query50	869	638	607	607
query51	4767	4639	4632	4632
query52	90	80	71	71
query53	238	184	174	174
query54	2645	2427	2484	2427
query55	87	87	81	81
query56	220	206	214	206
query57	1168	1102	1114	1102
query58	212	204	210	204
query59	3681	3359	3024	3024
query60	215	214	207	207
query61	95	93	94	93
query62	787	440	526	440
query63	194	177	176	176
query64	3574	1569	1499	1499
query65	3614	3586	3562	3562
query66	768	408	400	400
query67	16246	15767	15267	15267
query68	9081	663	644	644
query69	498	268	267	267
query70	1682	1386	1318	1318
query71	396	316	306	306
query72	6737	4864	4918	4864
query73	759	320	307	307
query74	6339	5809	5825	5809
query75	4801	3695	3676	3676
query76	4812	1151	1033	1033
query77	713	254	262	254
query78	12398	11788	11751	11751
query79	6266	629	629	629
query80	2162	374	381	374
query81	498	238	241	238
query82	1551	97	95	95
query83	163	134	131	131
query84	251	70	68	68
query85	1273	320	313	313
query86	350	291	295	291
query87	3236	2957	3072	2957
query88	4853	2333	2326	2326
query89	407	285	267	267
query90	1788	211	210	210
query91	155	124	121	121
query92	63	47	49	47
query93	4973	577	592	577
query94	758	211	216	211
query95	1960	1937	1935	1935
query96	648	323	342	323
query97	6394	6363	6314	6314
query98	217	219	189	189
query99	2825	883	993	883
Total cold run time: 314760 ms
Total hot run time: 211489 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.44 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1a46f621f371727af73e59461fbc177ad1cb595a, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.03
query3	0.25	0.04	0.05
query4	1.80	0.06	0.05
query5	0.54	0.52	0.52
query6	1.24	0.62	0.62
query7	0.02	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.48
query10	0.53	0.54	0.53
query11	0.12	0.09	0.09
query12	0.11	0.09	0.09
query13	0.62	0.61	0.62
query14	0.80	0.77	0.80
query15	0.78	0.76	0.77
query16	0.37	0.36	0.37
query17	1.02	0.99	1.01
query18	0.21	0.27	0.23
query19	1.83	1.87	1.82
query20	0.02	0.01	0.01
query21	15.50	0.56	0.55
query22	2.03	2.29	1.30
query23	16.94	0.98	1.06
query24	5.48	1.19	1.53
query25	0.37	0.10	0.06
query26	0.63	0.16	0.16
query27	0.04	0.04	0.03
query28	6.89	0.75	0.76
query29	12.66	2.31	2.08
query30	0.61	0.53	0.52
query31	2.83	0.39	0.38
query32	3.35	0.51	0.49
query33	3.07	3.04	3.06
query34	15.25	4.79	4.78
query35	4.88	4.84	4.84
query36	1.05	1.02	1.00
query37	0.05	0.05	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.15	0.14
query41	0.08	0.01	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.01
Total cold run time: 102.87 s
Total hot run time: 30.44 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 1a46f621f371727af73e59461fbc177ad1cb595a with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.8 seconds inserted 10000000 Rows, about 458K ops/s

@xiaokang
Copy link
Contributor Author

xiaokang commented Sep 7, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49099 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5975902984870d03e8fbfdafb3d3ae8e1943891c, data reload: false

------ Round 1 ----------------------------------
q1	18020	4356	4284	4284
q2	2073	155	141	141
q3	10475	1916	1885	1885
q4	10267	1229	1309	1229
q5	8791	3907	3941	3907
q6	223	124	122	122
q7	2030	1640	1602	1602
q8	9320	2717	2712	2712
q9	10474	9950	9860	9860
q10	8676	3542	3502	3502
q11	421	249	239	239
q12	463	300	298	298
q13	18370	4016	4034	4016
q14	359	335	333	333
q15	505	461	465	461
q16	545	448	449	448
q17	1134	948	936	936
q18	7204	6886	6810	6810
q19	1657	1531	1516	1516
q20	503	316	311	311
q21	4388	4093	4127	4093
q22	472	406	394	394
Total cold run time: 116370 ms
Total hot run time: 49099 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4315	4279	4287	4279
q2	318	230	229	229
q3	4136	4155	4167	4155
q4	2732	2747	2744	2744
q5	7256	7182	7102	7102
q6	234	119	117	117
q7	3239	2792	2845	2792
q8	4352	4451	4496	4451
q9	14281	14017	13975	13975
q10	4190	4276	4267	4267
q11	777	678	696	678
q12	1026	837	864	837
q13	6619	3759	3727	3727
q14	446	418	430	418
q15	497	463	460	460
q16	626	588	587	587
q17	3818	3840	3863	3840
q18	8858	8804	8850	8804
q19	1722	1645	1648	1645
q20	2410	2136	2099	2099
q21	8440	8508	8453	8453
q22	1042	947	900	900
Total cold run time: 81334 ms
Total hot run time: 76559 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 213444 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5975902984870d03e8fbfdafb3d3ae8e1943891c, data reload: false

query1	926	389	412	389
query2	6552	2222	2190	2190
query3	6924	203	205	203
query4	23435	21697	21793	21697
query5	19726	6546	6457	6457
query6	284	220	232	220
query7	4326	303	306	303
query8	256	276	254	254
query9	3051	2662	2588	2588
query10	466	308	318	308
query11	15826	14966	14937	14937
query12	132	78	75	75
query13	1023	445	439	439
query14	17821	13609	13788	13609
query15	375	223	232	223
query16	6483	282	262	262
query17	1898	956	892	892
query18	891	314	332	314
query19	209	155	154	154
query20	83	74	83	74
query21	191	102	94	94
query22	5227	5093	5058	5058
query23	34328	33717	33720	33717
query24	7702	6294	6252	6252
query25	537	431	427	427
query26	1268	162	164	162
query27	2322	296	289	289
query28	6105	2239	2219	2219
query29	2855	2744	2809	2744
query30	237	167	169	167
query31	965	729	739	729
query32	71	59	63	59
query33	456	259	270	259
query34	839	470	491	470
query35	1151	972	925	925
query36	1503	1320	1021	1021
query37	170	59	61	59
query38	3102	2960	2906	2906
query39	1375	1335	1346	1335
query40	304	94	97	94
query41	41	37	36	36
query42	82	86	82	82
query43	607	594	666	594
query44	1246	725	724	724
query45	248	230	230	230
query46	1227	957	942	942
query47	1812	1982	1761	1761
query48	524	415	432	415
query49	665	384	365	365
query50	851	631	647	631
query51	4774	4634	4714	4634
query52	91	76	78	76
query53	228	188	187	187
query54	2675	2465	2493	2465
query55	96	86	82	82
query56	244	202	224	202
query57	1276	1149	1063	1063
query58	223	219	212	212
query59	3529	3334	3216	3216
query60	226	227	218	218
query61	100	97	99	97
query62	827	435	479	435
query63	195	179	174	174
query64	3514	1620	1517	1517
query65	3607	3541	3559	3541
query66	774	404	422	404
query67	16087	15607	15371	15371
query68	7642	649	652	649
query69	497	266	276	266
query70	1512	1372	1478	1372
query71	406	302	309	302
query72	6824	5123	4808	4808
query73	742	325	325	325
query74	6342	5910	5829	5829
query75	4606	3741	3687	3687
query76	4442	1148	1199	1148
query77	552	256	258	256
query78	12882	11722	12028	11722
query79	6875	649	640	640
query80	2488	399	397	397
query81	522	241	241	241
query82	1756	99	97	97
query83	178	139	135	135
query84	259	70	74	70
query85	1397	323	326	323
query86	366	293	311	293
query87	3254	3061	3070	3061
query88	4785	2321	2346	2321
query89	413	303	302	302
query90	1754	218	220	218
query91	161	129	127	127
query92	62	52	50	50
query93	5248	572	545	545
query94	911	210	215	210
query95	2006	1990	1990	1990
query96	661	327	331	327
query97	6557	6405	6401	6401
query98	235	201	211	201
query99	2815	883	896	883
Total cold run time: 316314 ms
Total hot run time: 213444 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5975902984870d03e8fbfdafb3d3ae8e1943891c, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.04
query4	1.77	0.06	0.06
query5	0.53	0.52	0.52
query6	1.23	0.62	0.61
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.51	0.51	0.49
query10	0.53	0.53	0.54
query11	0.12	0.08	0.10
query12	0.12	0.10	0.09
query13	0.62	0.62	0.62
query14	0.78	0.78	0.78
query15	0.76	0.75	0.76
query16	0.39	0.36	0.40
query17	1.00	0.97	0.99
query18	0.23	0.23	0.26
query19	1.93	1.84	1.84
query20	0.02	0.01	0.01
query21	15.46	0.56	0.55
query22	2.07	2.94	1.48
query23	16.93	1.22	0.95
query24	5.06	2.01	1.69
query25	0.37	0.06	0.06
query26	0.74	0.16	0.15
query27	0.04	0.04	0.04
query28	6.41	0.73	0.75
query29	12.69	2.33	2.32
query30	0.62	0.54	0.56
query31	2.82	0.39	0.38
query32	3.36	0.50	0.49
query33	3.12	3.08	3.07
query34	15.25	4.82	4.82
query35	4.90	4.89	4.87
query36	1.07	1.00	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.15	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 102.2 s
Total hot run time: 31.43 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 5975902984870d03e8fbfdafb3d3ae8e1943891c with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.2 seconds inserted 10000000 Rows, about 450K ops/s

@xiaokang xiaokang merged commit d7aa336 into apache:branch-2.0 Sep 8, 2024
21 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/planner Issues or PRs related to the query planner kind/test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants