Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](cloud-merge) Fix the leak of cache fd #38051

Merged
merged 1 commit into from
Jul 22, 2024

Conversation

Lchangliang
Copy link
Contributor

If return if error, the fd cannot be removed. So we need to remove it first.

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Lchangliang
Copy link
Contributor Author

run buildall

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 40181 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 685a4377070e29efa8e0696d55ff6f4030044953, data reload: false

------ Round 1 ----------------------------------
q1	17737	4747	4361	4361
q2	2021	195	194	194
q3	10519	1219	1120	1120
q4	10226	824	695	695
q5	7601	2748	2707	2707
q6	226	142	139	139
q7	989	626	608	608
q8	9230	2102	2111	2102
q9	8839	6595	6678	6595
q10	8638	3836	3825	3825
q11	451	242	247	242
q12	391	227	229	227
q13	17751	3016	3012	3012
q14	272	234	243	234
q15	531	482	491	482
q16	494	375	379	375
q17	985	663	628	628
q18	8238	7592	7533	7533
q19	7732	1477	1347	1347
q20	684	325	321	321
q21	4998	3151	3209	3151
q22	350	283	289	283
Total cold run time: 118903 ms
Total hot run time: 40181 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4416	4284	4274	4274
q2	396	271	273	271
q3	3084	2974	2929	2929
q4	2031	1777	1722	1722
q5	5642	5594	5511	5511
q6	255	138	138	138
q7	2324	1877	1886	1877
q8	3293	3505	3460	3460
q9	8944	8983	8949	8949
q10	4166	3826	3868	3826
q11	614	496	523	496
q12	792	638	661	638
q13	15890	3198	3236	3198
q14	314	288	295	288
q15	520	497	490	490
q16	492	449	452	449
q17	1864	1562	1505	1505
q18	8249	7984	7852	7852
q19	1759	1735	1579	1579
q20	2106	1876	1877	1876
q21	8751	4860	4729	4729
q22	609	505	512	505
Total cold run time: 76511 ms
Total hot run time: 56562 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173845 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 685a4377070e29efa8e0696d55ff6f4030044953, data reload: false

query1	916	372	362	362
query2	6461	1968	1849	1849
query3	6658	207	211	207
query4	28419	17556	17260	17260
query5	3691	484	472	472
query6	263	157	159	157
query7	4578	293	280	280
query8	241	188	185	185
query9	8492	2462	2447	2447
query10	446	284	273	273
query11	11131	10087	10251	10087
query12	123	90	94	90
query13	1647	362	368	362
query14	10206	7556	7813	7556
query15	218	171	178	171
query16	7701	321	330	321
query17	1792	557	555	555
query18	1790	328	274	274
query19	193	149	148	148
query20	87	85	82	82
query21	205	132	127	127
query22	4508	4235	3940	3940
query23	34396	33680	34044	33680
query24	11274	2922	3020	2922
query25	651	385	387	385
query26	1533	158	152	152
query27	2918	277	270	270
query28	7798	2064	2058	2058
query29	1035	668	673	668
query30	259	159	155	155
query31	995	744	774	744
query32	97	55	53	53
query33	755	310	302	302
query34	982	495	508	495
query35	685	594	570	570
query36	1113	1005	976	976
query37	150	91	88	88
query38	3039	2851	2797	2797
query39	921	814	813	813
query40	253	118	117	117
query41	46	44	43	43
query42	117	99	99	99
query43	496	468	474	468
query44	1211	726	718	718
query45	193	164	168	164
query46	1092	737	727	727
query47	1855	1775	1797	1775
query48	378	295	288	288
query49	867	396	407	396
query50	780	397	391	391
query51	6811	6904	6778	6778
query52	109	99	96	96
query53	360	285	286	285
query54	923	446	443	443
query55	73	72	77	72
query56	287	260	286	260
query57	1151	1041	1082	1041
query58	257	241	247	241
query59	2986	2639	2706	2639
query60	301	277	271	271
query61	120	95	91	91
query62	797	646	650	646
query63	312	290	288	288
query64	10234	2195	1679	1679
query65	3385	3143	3106	3106
query66	1101	322	327	322
query67	15657	14970	15111	14970
query68	5252	535	542	535
query69	647	439	353	353
query70	1149	1135	1165	1135
query71	431	273	275	273
query72	7793	5534	5950	5534
query73	778	328	322	322
query74	6191	5686	5770	5686
query75	3541	2705	2713	2705
query76	3538	911	979	911
query77	621	315	294	294
query78	9640	9059	9116	9059
query79	3416	518	525	518
query80	2142	479	477	477
query81	595	222	219	219
query82	1292	137	137	137
query83	300	183	167	167
query84	279	87	148	87
query85	1430	313	301	301
query86	445	307	286	286
query87	3314	3105	3116	3105
query88	4327	2455	2470	2455
query89	479	410	404	404
query90	1666	201	198	198
query91	131	101	97	97
query92	62	49	53	49
query93	3633	509	518	509
query94	845	217	218	217
query95	411	323	332	323
query96	609	279	273	273
query97	3190	3017	3069	3017
query98	233	197	199	197
query99	1611	1255	1279	1255
Total cold run time: 290510 ms
Total hot run time: 173845 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 685a4377070e29efa8e0696d55ff6f4030044953, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.66	0.08	0.10
query5	0.50	0.50	0.50
query6	1.14	0.73	0.72
query7	0.02	0.01	0.02
query8	0.05	0.05	0.05
query9	0.54	0.50	0.48
query10	0.55	0.55	0.55
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.59	0.58	0.59
query14	0.77	0.79	0.79
query15	0.86	0.81	0.83
query16	0.36	0.36	0.37
query17	0.98	0.98	0.98
query18	0.23	0.22	0.23
query19	1.81	1.70	1.69
query20	0.02	0.01	0.01
query21	15.39	0.74	0.65
query22	4.11	6.95	2.35
query23	18.30	1.42	1.32
query24	2.06	0.27	0.22
query25	0.17	0.09	0.08
query26	0.30	0.21	0.20
query27	0.46	0.24	0.24
query28	13.20	1.02	1.00
query29	12.59	3.30	3.33
query30	0.24	0.06	0.05
query31	2.89	0.40	0.40
query32	3.26	0.50	0.48
query33	2.97	2.94	2.92
query34	17.14	4.35	4.37
query35	4.45	4.44	4.48
query36	0.66	0.47	0.48
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.04	0.03	0.03
query40	0.15	0.12	0.13
query41	0.09	0.05	0.05
query42	0.06	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.65 s
Total hot run time: 31.25 s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jul 18, 2024
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

PR approved by anyone and no changes requested.

@dataroaring dataroaring changed the title (cloud-merge) Fix the leak of cache fd [fix](cloud-merge) Fix the leak of cache fd Jul 18, 2024
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 2c91150 into apache:master Jul 22, 2024
29 of 33 checks passed
dataroaring pushed a commit that referenced this pull request Jul 24, 2024
If return if error, the fd cannot be removed. So we need to remove it
first.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants