Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](nereids)Add catalog/db/table filter info in SchemaScanNode #46864

Merged
merged 1 commit into from
Feb 6, 2025
Merged

[fix](nereids)Add catalog/db/table filter info in SchemaScanNode #46864

merged 1 commit into from
Feb 6, 2025

Conversation

Jibing-Li
Copy link
Contributor

@Jibing-Li Jibing-Li commented Jan 13, 2025 •
edited
Loading

What problem does this PR solve?

While querying information_schema.tables, we need to filter the catalogs/dbs/tables if user specify them thought where statement. Before, we didn't pass the query predicate to BE. This pr pass the predicate to BE and filter the catalog/db/table in FrontendServiceImpl.listTableStatus

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Jibing-Li Jibing-Li marked this pull request as ready for review January 13, 2025 03:23
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32658 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fe02861c74d0aa83e9c0d954895af1e0ed75da8a, data reload: false

------ Round 1 ----------------------------------
q1	17627	6220	6075	6075
q2	2047	319	171	171
q3	10579	1337	725	725
q4	10288	895	431	431
q5	8734	2175	1961	1961
q6	204	183	150	150
q7	905	743	636	636
q8	9243	1358	1244	1244
q9	6034	5030	4881	4881
q10	6798	2301	1830	1830
q11	488	281	260	260
q12	344	362	220	220
q13	17774	3634	3086	3086
q14	240	228	218	218
q15	576	513	498	498
q16	639	628	599	599
q17	582	860	329	329
q18	7017	6480	6315	6315
q19	1547	965	539	539
q20	310	321	192	192
q21	2811	2192	1987	1987
q22	375	331	311	311
Total cold run time: 105162 ms
Total hot run time: 32658 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6267	6212	6225	6212
q2	248	335	242	242
q3	2292	2657	2353	2353
q4	1392	1789	1365	1365
q5	4363	4768	5018	4768
q6	182	177	138	138
q7	2101	1979	1856	1856
q8	2644	2778	2682	2682
q9	7356	7316	7185	7185
q10	3063	3361	2809	2809
q11	601	540	491	491
q12	675	750	609	609
q13	3467	3835	3232	3232
q14	290	322	291	291
q15	564	515	514	514
q16	654	686	645	645
q17	1226	1740	1264	1264
q18	7796	7480	7369	7369
q19	810	1005	1181	1005
q20	1950	2089	1970	1970
q21	5719	5136	4991	4991
q22	617	640	575	575
Total cold run time: 54277 ms
Total hot run time: 52566 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188654 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fe02861c74d0aa83e9c0d954895af1e0ed75da8a, data reload: false

query1	989	387	378	378
query2	6514	2407	2342	2342
query3	6712	215	216	215
query4	33568	23476	23595	23476
query5	4385	634	491	491
query6	299	207	174	174
query7	4633	502	311	311
query8	295	237	224	224
query9	9602	2684	2670	2670
query10	462	315	246	246
query11	17820	15208	15085	15085
query12	157	105	104	104
query13	1628	510	384	384
query14	9688	7367	7076	7076
query15	242	192	187	187
query16	8266	589	448	448
query17	1594	781	550	550
query18	2101	395	299	299
query19	215	177	149	149
query20	114	108	112	108
query21	209	120	99	99
query22	4392	4408	4240	4240
query23	34740	33095	33086	33086
query24	6708	2206	2229	2206
query25	508	461	369	369
query26	1197	279	154	154
query27	2065	459	329	329
query28	5163	2417	2395	2395
query29	721	546	408	408
query30	233	185	156	156
query31	976	859	767	767
query32	82	59	62	59
query33	513	350	295	295
query34	763	845	519	519
query35	830	798	723	723
query36	1013	1059	931	931
query37	126	99	79	79
query38	4165	4002	3976	3976
query39	1486	1441	1406	1406
query40	208	123	107	107
query41	59	59	65	59
query42	128	106	155	106
query43	514	539	505	505
query44	1358	814	808	808
query45	176	169	165	165
query46	851	1047	646	646
query47	1816	1807	1767	1767
query48	391	400	308	308
query49	776	496	412	412
query50	632	699	387	387
query51	6989	7018	6760	6760
query52	104	106	96	96
query53	222	254	194	194
query54	478	494	412	412
query55	83	77	78	77
query56	259	283	241	241
query57	1184	1149	1100	1100
query58	246	227	246	227
query59	3048	3102	3011	3011
query60	279	256	250	250
query61	121	116	130	116
query62	807	768	688	688
query63	221	194	193	193
query64	4452	1061	656	656
query65	3261	3152	3194	3152
query66	863	424	329	329
query67	15862	15750	15465	15465
query68	7782	701	515	515
query69	466	303	271	271
query70	1187	1159	1099	1099
query71	401	281	266	266
query72	6237	3853	3909	3853
query73	640	749	348	348
query74	10374	9074	8933	8933
query75	3269	3146	2658	2658
query76	3273	1168	784	784
query77	538	373	275	275
query78	9908	9838	9305	9305
query79	2994	795	609	609
query80	644	508	447	447
query81	484	279	238	238
query82	609	142	126	126
query83	173	171	158	158
query84	241	98	73	73
query85	784	369	295	295
query86	359	311	300	300
query87	4367	4400	4262	4262
query88	4774	2143	2120	2120
query89	458	327	292	292
query90	1823	193	190	190
query91	135	136	130	130
query92	71	58	53	53
query93	1687	871	541	541
query94	695	406	295	295
query95	330	272	255	255
query96	489	625	281	281
query97	2899	2829	2764	2764
query98	229	203	194	194
query99	1476	1555	1373	1373
Total cold run time: 290358 ms
Total hot run time: 188654 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.85 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fe02861c74d0aa83e9c0d954895af1e0ed75da8a, data reload: false

query1	0.03	0.04	0.04
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.11	0.10
query5	0.43	0.42	0.40
query6	1.13	0.66	0.65
query7	0.03	0.01	0.01
query8	0.04	0.04	0.04
query9	0.58	0.50	0.49
query10	0.56	0.56	0.57
query11	0.14	0.11	0.10
query12	0.14	0.11	0.11
query13	0.61	0.60	0.60
query14	2.79	2.72	2.83
query15	0.89	0.82	0.83
query16	0.40	0.39	0.37
query17	1.05	1.06	1.04
query18	0.22	0.22	0.21
query19	2.00	1.83	2.01
query20	0.01	0.01	0.02
query21	15.36	0.92	0.58
query22	0.75	0.80	0.64
query23	15.28	1.42	0.55
query24	3.03	1.52	1.62
query25	0.14	0.10	0.22
query26	0.29	0.13	0.14
query27	0.04	0.07	0.05
query28	14.45	1.45	1.04
query29	12.60	3.99	3.33
query30	0.25	0.09	0.06
query31	2.81	0.61	0.38
query32	3.22	0.55	0.45
query33	3.08	3.18	3.11
query34	16.79	5.05	4.49
query35	4.49	4.47	4.44
query36	0.64	0.48	0.47
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.12	0.13
query41	0.07	0.02	0.02
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 106.67 s
Total hot run time: 31.85 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 40.35% (10514/26054)
Line Coverage: 31.10% (89086/286448)
Region Coverage: 30.22% (45562/150755)
Branch Coverage: 26.51% (23153/87344)
Coverage Report: http://coverage.selectdb-in.cc/coverage/fe02861c74d0aa83e9c0d954895af1e0ed75da8a_fe02861c74d0aa83e9c0d954895af1e0ed75da8a/report/index.html

@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32502 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c7ba9a763af772cc2751fbd3f47bae81b8cca056, data reload: false

------ Round 1 ----------------------------------
q1	17594	6085	6012	6012
q2	2055	299	168	168
q3	10414	1264	712	712
q4	10216	873	450	450
q5	7538	2155	1941	1941
q6	209	176	149	149
q7	893	741	596	596
q8	9257	1362	1159	1159
q9	5130	4972	4855	4855
q10	6750	2288	1845	1845
q11	472	280	261	261
q12	341	357	220	220
q13	17786	3701	3027	3027
q14	227	232	216	216
q15	585	523	505	505
q16	616	622	591	591
q17	562	840	327	327
q18	6943	6540	6467	6467
q19	2445	965	551	551
q20	288	310	192	192
q21	2794	2152	1961	1961
q22	366	332	297	297
Total cold run time: 103481 ms
Total hot run time: 32502 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6305	6186	6236	6186
q2	238	332	240	240
q3	2263	2655	2349	2349
q4	1407	1776	1351	1351
q5	4324	4733	4635	4635
q6	179	172	139	139
q7	1959	1871	1718	1718
q8	2511	2706	2617	2617
q9	6920	6899	6861	6861
q10	2909	3203	2677	2677
q11	579	513	487	487
q12	656	722	572	572
q13	3251	3631	3116	3116
q14	297	297	262	262
q15	583	496	494	494
q16	643	688	655	655
q17	1171	1681	1232	1232
q18	7303	7216	7064	7064
q19	773	1142	1043	1043
q20	1940	1911	1822	1822
q21	5560	4984	4720	4720
q22	593	626	587	587
Total cold run time: 52364 ms
Total hot run time: 50827 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 188108 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c7ba9a763af772cc2751fbd3f47bae81b8cca056, data reload: false

query1	980	389	360	360
query2	6524	2406	2289	2289
query3	6710	214	219	214
query4	33939	23888	22995	22995
query5	4410	612	456	456
query6	295	192	176	176
query7	4621	493	304	304
query8	283	231	224	224
query9	9551	2727	2721	2721
query10	472	309	265	265
query11	18285	15734	14946	14946
query12	170	106	104	104
query13	1636	497	382	382
query14	11488	6405	6926	6405
query15	218	207	191	191
query16	8053	609	410	410
query17	1548	730	540	540
query18	2071	399	297	297
query19	235	178	146	146
query20	116	121	110	110
query21	210	119	98	98
query22	4202	4393	4343	4343
query23	34254	33730	33349	33349
query24	6302	2307	2253	2253
query25	477	481	415	415
query26	1157	268	162	162
query27	1979	461	335	335
query28	5246	2438	2414	2414
query29	576	563	431	431
query30	248	184	164	164
query31	963	857	810	810
query32	82	65	63	63
query33	559	338	309	309
query34	732	838	494	494
query35	810	831	721	721
query36	1014	1011	945	945
query37	128	99	77	77
query38	4024	4229	4125	4125
query39	1469	1443	1411	1411
query40	206	115	104	104
query41	52	54	48	48
query42	125	104	103	103
query43	515	537	478	478
query44	1297	804	812	804
query45	182	171	170	170
query46	846	1031	638	638
query47	1800	1821	1777	1777
query48	364	398	317	317
query49	788	476	401	401
query50	622	637	386	386
query51	6934	6761	6748	6748
query52	108	104	91	91
query53	223	248	182	182
query54	479	480	410	410
query55	79	79	78	78
query56	253	288	250	250
query57	1144	1152	1070	1070
query58	264	232	233	232
query59	2874	3232	3019	3019
query60	286	272	251	251
query61	121	117	123	117
query62	864	739	719	719
query63	225	191	189	189
query64	4057	1029	657	657
query65	3202	3153	3194	3153
query66	1042	416	314	314
query67	16015	15689	15457	15457
query68	7388	710	517	517
query69	449	298	258	258
query70	1149	1029	1137	1029
query71	439	287	251	251
query72	6114	3881	3878	3878
query73	652	749	355	355
query74	10163	9076	9001	9001
query75	3905	3033	2657	2657
query76	3358	1177	777	777
query77	754	366	279	279
query78	9979	10041	9464	9464
query79	3346	818	576	576
query80	644	550	448	448
query81	493	273	235	235
query82	673	161	119	119
query83	243	175	154	154
query84	238	88	71	71
query85	753	348	314	314
query86	416	313	301	301
query87	4475	4413	4275	4275
query88	4943	2154	2168	2154
query89	403	326	300	300
query90	1867	191	193	191
query91	139	146	108	108
query92	70	59	55	55
query93	1802	890	533	533
query94	661	403	296	296
query95	342	266	257	257
query96	500	608	274	274
query97	2847	2966	2805	2805
query98	241	200	201	200
query99	1586	1527	1359	1359
Total cold run time: 292036 ms
Total hot run time: 188108 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.99 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c7ba9a763af772cc2751fbd3f47bae81b8cca056, data reload: false

query1	0.03	0.04	0.03
query2	0.07	0.03	0.03
query3	0.24	0.07	0.07
query4	1.61	0.11	0.10
query5	0.44	0.41	0.38
query6	1.16	0.66	0.64
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.56	0.53	0.51
query10	0.54	0.54	0.56
query11	0.15	0.11	0.10
query12	0.14	0.10	0.11
query13	0.60	0.60	0.60
query14	2.83	2.88	2.86
query15	0.89	0.83	0.83
query16	0.38	0.37	0.38
query17	0.98	0.96	1.04
query18	0.24	0.22	0.20
query19	1.83	1.82	1.98
query20	0.01	0.02	0.01
query21	15.35	0.97	0.59
query22	0.74	0.80	0.56
query23	15.38	1.47	0.55
query24	4.00	1.76	1.78
query25	0.18	0.18	0.18
query26	0.21	0.14	0.13
query27	0.06	0.07	0.04
query28	14.12	1.42	1.04
query29	12.55	3.87	3.23
query30	0.25	0.09	0.06
query31	2.83	0.61	0.38
query32	3.24	0.54	0.46
query33	3.12	3.08	3.09
query34	16.78	5.09	4.51
query35	4.47	4.42	4.52
query36	0.79	0.49	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.17	0.13	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 107.34 s
Total hot run time: 31.99 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 40.35% (10512/26055)
Line Coverage: 31.10% (89095/286442)
Region Coverage: 30.21% (45550/150756)
Branch Coverage: 26.50% (23149/87344)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c7ba9a763af772cc2751fbd3f47bae81b8cca056_c7ba9a763af772cc2751fbd3f47bae81b8cca056/report/index.html

@Jibing-Li
Copy link
Contributor Author

run external

@Jibing-Li
Copy link
Contributor Author

run p0

@dataroaring dataroaring added usercase Important user case type label dev/3.0.x labels Jan 14, 2025
@morrySnow morrySnow requested a review from Gabriel39 January 14, 2025 06:28
@Jibing-Li Jibing-Li force-pushed the informationschema branch 2 times, most recently from b78ad94 to 9078a4f Compare January 14, 2025 10:00
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32456 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9078a4f7b64fcbcbb50f002794b70344c29473eb, data reload: false

------ Round 1 ----------------------------------
q1	17569	6246	6037	6037
q2	2048	301	167	167
q3	10432	1206	734	734
q4	10212	857	428	428
q5	7520	2197	1966	1966
q6	202	176	146	146
q7	877	788	598	598
q8	9228	1331	1104	1104
q9	5169	4899	4944	4899
q10	6950	2326	1899	1899
q11	474	272	255	255
q12	333	354	217	217
q13	17756	3664	3073	3073
q14	229	251	217	217
q15	588	532	513	513
q16	624	620	583	583
q17	566	845	323	323
q18	6761	6302	6391	6302
q19	2808	933	528	528
q20	293	319	186	186
q21	2763	2094	1965	1965
q22	363	327	316	316
Total cold run time: 103765 ms
Total hot run time: 32456 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6294	6199	6200	6199
q2	236	326	244	244
q3	2264	2632	2365	2365
q4	1379	1816	1362	1362
q5	4312	4743	4746	4743
q6	191	180	138	138
q7	2049	1976	1806	1806
q8	2640	2823	2693	2693
q9	7314	7282	7341	7282
q10	3074	3287	2712	2712
q11	586	526	482	482
q12	674	800	602	602
q13	3536	3877	3337	3337
q14	287	302	297	297
q15	580	537	524	524
q16	667	686	655	655
q17	1212	1714	1266	1266
q18	7737	7495	7452	7452
q19	796	817	1133	817
q20	2025	2062	1889	1889
q21	5533	5057	5016	5016
q22	618	620	560	560
Total cold run time: 54004 ms
Total hot run time: 52441 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 40.57% (10573/26062)
Line Coverage: 31.24% (89497/286441)
Region Coverage: 30.36% (45777/150767)
Branch Coverage: 26.65% (23281/87346)
Coverage Report: http://coverage.selectdb-in.cc/coverage/9078a4f7b64fcbcbb50f002794b70344c29473eb_9078a4f7b64fcbcbb50f002794b70344c29473eb/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 195510 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9078a4f7b64fcbcbb50f002794b70344c29473eb, data reload: false

query1	1315	986	942	942
query2	6360	2088	2034	2034
query3	11118	4642	4595	4595
query4	39877	23588	23586	23586
query5	5197	604	448	448
query6	281	198	189	189
query7	3644	487	320	320
query8	300	248	242	242
query9	6377	2738	2747	2738
query10	428	304	265	265
query11	16045	14957	15227	14957
query12	156	104	105	104
query13	946	527	414	414
query14	10655	7430	6879	6879
query15	231	200	188	188
query16	7907	645	509	509
query17	1579	730	580	580
query18	2126	404	305	305
query19	197	189	168	168
query20	121	117	111	111
query21	219	119	99	99
query22	4487	4476	4685	4476
query23	34681	33787	34253	33787
query24	6344	2284	2350	2284
query25	493	490	386	386
query26	763	258	153	153
query27	2143	473	339	339
query28	5201	2493	2507	2493
query29	572	556	443	443
query30	221	219	153	153
query31	972	913	821	821
query32	76	62	57	57
query33	480	352	315	315
query34	746	865	524	524
query35	808	811	759	759
query36	1010	1037	964	964
query37	119	100	80	80
query38	4254	4292	4235	4235
query39	1473	1436	1431	1431
query40	203	121	112	112
query41	54	51	73	51
query42	128	108	103	103
query43	507	529	498	498
query44	1370	837	844	837
query45	192	174	163	163
query46	872	1057	653	653
query47	1940	1952	1880	1880
query48	380	409	332	332
query49	715	491	386	386
query50	673	672	396	396
query51	6956	7071	6983	6983
query52	100	103	96	96
query53	228	250	191	191
query54	483	516	432	432
query55	87	86	91	86
query56	274	275	262	262
query57	1234	1218	1136	1136
query58	266	246	290	246
query59	3198	3296	3148	3148
query60	292	277	261	261
query61	133	111	116	111
query62	913	777	698	698
query63	225	184	181	181
query64	3462	1021	681	681
query65	3216	3149	3171	3149
query66	707	401	314	314
query67	15677	15708	15345	15345
query68	4912	834	530	530
query69	467	289	252	252
query70	1201	1141	1122	1122
query71	381	296	261	261
query72	6401	3896	3840	3840
query73	765	746	363	363
query74	10154	9060	8824	8824
query75	3172	3165	2659	2659
query76	3535	1169	769	769
query77	479	384	292	292
query78	10131	10178	9246	9246
query79	3159	837	600	600
query80	1799	563	436	436
query81	587	274	232	232
query82	368	157	115	115
query83	277	173	161	161
query84	247	99	74	74
query85	778	351	309	309
query86	486	316	305	305
query87	4451	4515	4482	4482
query88	4537	2185	2166	2166
query89	406	337	293	293
query90	1821	186	187	186
query91	139	134	112	112
query92	70	58	54	54
query93	2826	880	541	541
query94	721	409	297	297
query95	339	266	250	250
query96	501	610	288	288
query97	2805	2839	2789	2789
query98	231	197	193	193
query99	1394	1501	1365	1365
Total cold run time: 293947 ms
Total hot run time: 195510 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.1 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9078a4f7b64fcbcbb50f002794b70344c29473eb, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.03
query3	0.24	0.07	0.07
query4	1.61	0.10	0.10
query5	0.41	0.42	0.43
query6	1.15	0.65	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.50	0.50
query10	0.56	0.57	0.56
query11	0.13	0.10	0.10
query12	0.14	0.11	0.10
query13	0.61	0.60	0.59
query14	2.70	2.74	2.72
query15	0.89	0.83	0.83
query16	0.36	0.37	0.38
query17	1.07	1.04	1.03
query18	0.23	0.20	0.22
query19	1.90	1.72	2.03
query20	0.01	0.01	0.01
query21	15.37	0.93	0.57
query22	0.76	0.77	0.61
query23	15.37	1.51	0.61
query24	3.00	1.51	0.81
query25	0.21	0.16	0.09
query26	0.34	0.15	0.14
query27	0.05	0.05	0.05
query28	13.25	1.52	1.05
query29	12.55	3.83	3.19
query30	0.26	0.09	0.07
query31	2.81	0.60	0.39
query32	3.22	0.55	0.46
query33	3.09	3.02	3.04
query34	16.88	5.21	4.53
query35	4.59	4.62	4.66
query36	0.63	0.49	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.04	0.03	0.02
query40	0.18	0.13	0.12
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.66 s
Total hot run time: 31.1 s

@Jibing-Li
Copy link
Contributor Author

run external

1 similar comment
@Jibing-Li
Copy link
Contributor Author

run external

Gabriel39
Gabriel39 previously approved these changes Jan 21, 2025
View reviewed changes
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Feb 6, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 6, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 32048 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1cff65f68a3e6653f573548e8e102dce1250339e, data reload: false

------ Round 1 ----------------------------------
q1	17585	5487	5377	5377
q2	2059	305	169	169
q3	10487	1238	719	719
q4	10221	961	523	523
q5	7559	2420	2159	2159
q6	193	168	134	134
q7	904	740	588	588
q8	9225	1347	1145	1145
q9	5165	4957	4828	4828
q10	6868	2349	1905	1905
q11	460	270	254	254
q12	335	356	220	220
q13	17788	3720	3056	3056
q14	237	241	208	208
q15	519	482	464	464
q16	642	611	583	583
q17	572	874	324	324
q18	7269	6567	6429	6429
q19	1681	960	536	536
q20	317	319	187	187
q21	2802	2190	1938	1938
q22	362	330	302	302
Total cold run time: 103250 ms
Total hot run time: 32048 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5539	5498	5513	5498
q2	235	327	230	230
q3	2288	2688	2333	2333
q4	1422	1816	1410	1410
q5	4309	4765	4647	4647
q6	168	159	127	127
q7	2079	1936	1792	1792
q8	2602	2854	2697	2697
q9	7417	7315	7306	7306
q10	3059	3274	2795	2795
q11	584	507	480	480
q12	633	774	574	574
q13	3462	3911	3272	3272
q14	275	297	298	297
q15	517	466	482	466
q16	656	697	647	647
q17	1238	1761	1270	1270
q18	7722	7445	7366	7366
q19	796	1092	1109	1092
q20	1983	2045	1896	1896
q21	5886	5165	4970	4970
q22	619	636	566	566
Total cold run time: 53489 ms
Total hot run time: 51731 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192302 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1cff65f68a3e6653f573548e8e102dce1250339e, data reload: false

query1	1306	933	957	933
query2	6237	2049	2036	2036
query3	11120	4753	4286	4286
query4	61314	32662	23556	23556
query5	5221	605	456	456
query6	434	217	195	195
query7	5548	510	299	299
query8	347	255	227	227
query9	8219	2588	2602	2588
query10	452	318	255	255
query11	15969	15391	15681	15391
query12	163	113	117	113
query13	1448	570	427	427
query14	11308	7407	6636	6636
query15	220	211	188	188
query16	7189	657	495	495
query17	1138	748	603	603
query18	1747	403	322	322
query19	208	199	195	195
query20	131	116	111	111
query21	213	118	107	107
query22	4301	4718	4580	4580
query23	34187	33589	33394	33394
query24	5450	2351	2323	2323
query25	464	459	419	419
query26	658	276	157	157
query27	1898	488	326	326
query28	3930	2491	2427	2427
query29	536	563	428	428
query30	221	199	161	161
query31	953	910	801	801
query32	70	58	58	58
query33	459	346	298	298
query34	736	897	510	510
query35	830	834	769	769
query36	1008	1056	963	963
query37	122	110	76	76
query38	4406	4357	4198	4198
query39	1503	1455	1482	1455
query40	212	117	112	112
query41	58	49	50	49
query42	121	109	102	102
query43	533	551	500	500
query44	1456	853	822	822
query45	177	181	169	169
query46	875	1061	655	655
query47	1908	1919	1848	1848
query48	405	401	321	321
query49	740	488	383	383
query50	665	668	397	397
query51	4308	4303	4294	4294
query52	109	104	94	94
query53	240	264	200	200
query54	501	507	418	418
query55	81	83	78	78
query56	263	275	255	255
query57	1193	1198	1132	1132
query58	247	235	247	235
query59	3237	3216	3172	3172
query60	272	274	260	260
query61	120	116	123	116
query62	756	725	657	657
query63	224	193	187	187
query64	1273	1020	658	658
query65	3238	3155	3157	3155
query66	684	395	367	367
query67	16073	15812	15531	15531
query68	5057	823	545	545
query69	497	298	261	261
query70	1158	1130	1044	1044
query71	424	290	263	263
query72	6104	3935	3861	3861
query73	796	753	360	360
query74	9872	8874	9086	8874
query75	3293	3198	2677	2677
query76	3800	1187	758	758
query77	472	363	277	277
query78	10046	10030	9264	9264
query79	2790	807	626	626
query80	1689	523	448	448
query81	560	281	233	233
query82	355	152	122	122
query83	276	167	208	167
query84	296	85	72	72
query85	781	347	323	323
query86	450	308	302	302
query87	4415	4460	4501	4460
query88	3419	2168	2146	2146
query89	403	331	295	295
query90	1619	192	193	192
query91	131	209	109	109
query92	66	59	52	52
query93	2387	896	544	544
query94	770	421	310	310
query95	336	273	257	257
query96	498	624	297	297
query97	2907	2851	2769	2769
query98	230	209	197	197
query99	1270	1345	1268	1268
Total cold run time: 308800 ms
Total hot run time: 192302 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.09 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1cff65f68a3e6653f573548e8e102dce1250339e, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.24	0.05	0.06
query4	1.64	0.08	0.09
query5	0.43	0.41	0.40
query6	1.14	0.67	0.66
query7	0.02	0.02	0.02
query8	0.05	0.05	0.05
query9	0.56	0.48	0.50
query10	0.56	0.55	0.55
query11	0.16	0.11	0.12
query12	0.16	0.13	0.13
query13	0.62	0.60	0.59
query14	2.74	2.97	2.86
query15	0.90	0.84	0.85
query16	0.40	0.40	0.38
query17	1.04	1.07	0.98
query18	0.21	0.20	0.19
query19	1.95	2.02	1.82
query20	0.01	0.01	0.01
query21	15.37	0.97	0.66
query22	0.76	0.80	0.70
query23	14.96	1.51	0.68
query24	2.19	0.36	0.22
query25	0.14	0.10	0.08
query26	0.28	0.19	0.18
query27	0.08	0.08	0.08
query28	13.44	1.28	0.56
query29	12.62	4.07	3.39
query30	0.25	0.08	0.05
query31	2.84	0.60	0.40
query32	3.23	0.58	0.49
query33	2.95	3.04	3.05
query34	16.49	5.23	4.56
query35	4.64	4.64	4.65
query36	0.81	0.50	0.49
query37	0.20	0.16	0.16
query38	0.15	0.14	0.15
query39	0.06	0.05	0.04
query40	0.17	0.13	0.14
query41	0.09	0.05	0.06
query42	0.06	0.04	0.05
query43	0.05	0.05	0.04
Total cold run time: 104.78 s
Total hot run time: 31.09 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 42.08% (11014/26171)
Line Coverage: 32.36% (92985/287367)
Region Coverage: 31.51% (47674/151276)
Branch Coverage: 27.52% (24117/87628)
Coverage Report: http://coverage.selectdb-in.cc/coverage/1cff65f68a3e6653f573548e8e102dce1250339e_1cff65f68a3e6653f573548e8e102dce1250339e/report/index.html

@Jibing-Li
Copy link
Contributor Author

run cloud_p0

1 similar comment
@Jibing-Li
Copy link
Contributor Author

run cloud_p0

@Jibing-Li Jibing-Li merged commit 986fbc4 into apache:master Feb 6, 2025
26 of 28 checks passed
@Jibing-Li Jibing-Li deleted the informationschema branch February 6, 2025 08:00
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Feb 6, 2025
@Jibing-Li
[fix](nereids)Add catalog/db/table filter info in SchemaScanNode (apa…
6986c0e 
…che#46864)

While querying information_schema.tables, we need to filter the
catalogs/dbs/tables if user specify them thought where statement.
Before, we didn't pass the query predicate to BE. This pr pass the
predicate to BE and filter the catalog/db/table in
FrontendServiceImpl.listTableStatus
@Jibing-Li Jibing-Li mentioned this pull request Feb 6, 2025
Merged
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Feb 7, 2025
@Jibing-Li
[fix](nereids)Add catalog/db/table filter info in SchemaScanNode (apa…
411e1d7 
…che#46864)

While querying information_schema.tables, we need to filter the
catalogs/dbs/tables if user specify them thought where statement.
Before, we didn't pass the query predicate to BE. This pr pass the
predicate to BE and filter the catalog/db/table in
FrontendServiceImpl.listTableStatus
gavinchou pushed a commit that referenced this pull request Feb 8, 2025
@Jibing-Li
[fix](nereids)Add catalog/db/table filter info in SchemaScanNode (#46864
a733a56 
) (#47550)

backport: #46864
Jibing-Li added a commit to Jibing-Li/incubator-doris that referenced this pull request Feb 11, 2025
@Jibing-Li
[fix](nereids)Add catalog/db/table filter info in SchemaScanNode (apa…
462de88 
…che#46864)

While querying information_schema.tables, we need to filter the
catalogs/dbs/tables if user specify them thought where statement.
Before, we didn't pass the query predicate to BE. This pr pass the
predicate to BE and filter the catalog/db/table in
FrontendServiceImpl.listTableStatus
@Jibing-Li Jibing-Li mentioned this pull request Feb 11, 2025
Merged
Jibing-Li added a commit that referenced this pull request Feb 11, 2025
@Jibing-Li
[fix](nereids)Add catalog/db/table filter info in SchemaScanNode (#46864
e2f9b43 
) (#47759)

backport: #46864
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
@Jibing-Li @lzyy2024
[fix](nereids)Add catalog/db/table filter info in SchemaScanNode (apa…
65e7d45 
…che#46864)

### What problem does this PR solve?

While querying information_schema.tables, we need to filter the
catalogs/dbs/tables if user specify them thought where statement.
Before, we didn't pass the query predicate to BE. This pr pass the
predicate to BE and filter the catalog/db/table in
FrontendServiceImpl.listTableStatus
hubgeter pushed a commit to hubgeter/doris that referenced this pull request Mar 12, 2025
@Jibing-Li
[fix](nereids)Add catalog/db/table filter info in SchemaScanNode (apa…
b2b21ac 
…che#46864) (apache#3747)

While querying information_schema.tables, we need to filter the
catalogs/dbs/tables if user specify them thought where statement.
Before, we didn't pass the query predicate to BE. This pr pass the
predicate to BE and filter the catalog/db/table in
FrontendServiceImpl.listTableStatus

backport: apache#46864
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Gabriel39 Gabriel39 Gabriel39 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.4-merged reviewed usercase Important user case type label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@Jibing-Li @hello-stephen @doris-robot @Gabriel39 @morrySnow @gavinchou @dataroaring