Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[bugfix](backup)(cooldown) cancel backup properly when be backup failed #38724 #38994

Merged
merged 1 commit into from
Aug 7, 2024
Merged

[bugfix](backup)(cooldown) cancel backup properly when be backup failed #38724 #38994

merged 1 commit into from
Aug 7, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Aug 7, 2024

cherry pick from #38724

@w41ter
Copy link
Contributor Author

w41ter commented Aug 7, 2024

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49928 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 5ac5c02c299d3289b0b9a92c015708349377cc19, data reload: false

------ Round 1 ----------------------------------
q1	18202	4396	4322	4322
q2	2071	162	147	147
q3	10257	1884	1910	1884
q4	10381	1310	1333	1310
q5	8950	3958	3912	3912
q6	242	126	148	126
q7	2039	1630	1628	1628
q8	9295	2743	2708	2708
q9	10665	10239	10195	10195
q10	8629	3522	3528	3522
q11	426	237	250	237
q12	470	302	312	302
q13	18337	3966	3990	3966
q14	361	337	326	326
q15	512	463	470	463
q16	691	566	569	566
q17	1128	980	984	980
q18	7443	6847	6987	6847
q19	1805	1694	1634	1634
q20	562	317	304	304
q21	4422	4137	4103	4103
q22	524	458	446	446
Total cold run time: 117412 ms
Total hot run time: 49928 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4326	4359	4280	4280
q2	325	228	217	217
q3	4191	4110	4176	4110
q4	2755	2761	2777	2761
q5	7129	7124	7059	7059
q6	237	118	118	118
q7	3214	2864	2870	2864
q8	4410	4480	4526	4480
q9	16869	16769	16782	16769
q10	4235	4262	4327	4262
q11	783	705	678	678
q12	1037	861	849	849
q13	6901	3767	3703	3703
q14	448	424	424	424
q15	493	461	465	461
q16	753	683	685	683
q17	3768	3880	3958	3880
q18	8794	8754	8714	8714
q19	1761	1723	1675	1675
q20	2360	2162	2122	2122
q21	8663	8573	8501	8501
q22	1079	956	961	956
Total cold run time: 84531 ms
Total hot run time: 79566 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.74% (8102/21467)
Line Coverage: 29.38% (66354/225881)
Region Coverage: 28.88% (34242/118564)
Branch Coverage: 24.77% (17597/71052)
Coverage Report: http://coverage.selectdb-in.cc/coverage/5ac5c02c299d3289b0b9a92c015708349377cc19_5ac5c02c299d3289b0b9a92c015708349377cc19/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 203516 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 5ac5c02c299d3289b0b9a92c015708349377cc19, data reload: false

query1	950	400	412	400
query2	6573	2812	2753	2753
query3	6920	214	204	204
query4	20848	18135	17981	17981
query5	19723	6546	6532	6532
query6	304	233	227	227
query7	4335	297	312	297
query8	445	417	439	417
query9	3107	2676	2635	2635
query10	407	303	286	286
query11	11369	10763	10724	10724
query12	130	80	76	76
query13	5608	716	715	715
query14	18371	13697	13284	13284
query15	371	245	253	245
query16	6442	290	265	265
query17	1698	1442	894	894
query18	2298	414	412	412
query19	207	154	154	154
query20	79	77	77	77
query21	191	107	90	90
query22	5334	4843	4950	4843
query23	32447	31726	31997	31726
query24	6975	6559	6500	6500
query25	529	423	435	423
query26	528	162	160	160
query27	1889	303	295	295
query28	6160	2395	2354	2354
query29	2848	2764	2690	2690
query30	251	170	164	164
query31	923	747	723	723
query32	71	63	65	63
query33	395	248	248	248
query34	844	470	486	470
query35	1136	856	929	856
query36	1476	1102	1209	1102
query37	92	62	59	59
query38	3077	2942	2920	2920
query39	1390	1331	1335	1331
query40	208	97	95	95
query41	45	43	44	43
query42	88	81	85	81
query43	725	702	742	702
query44	1168	742	745	742
query45	253	237	241	237
query46	1228	974	991	974
query47	1896	1786	1676	1676
query48	1026	745	709	709
query49	629	382	374	374
query50	875	616	612	612
query51	4795	4630	4690	4630
query52	95	84	75	75
query53	457	319	320	319
query54	2681	2468	2462	2462
query55	82	96	92	92
query56	236	220	206	206
query57	1163	1215	1114	1114
query58	216	215	195	195
query59	4079	3953	3939	3939
query60	228	208	202	202
query61	99	101	104	101
query62	867	491	447	447
query63	488	345	341	341
query64	2646	1514	1401	1401
query65	3611	3524	3589	3524
query66	775	382	399	382
query67	15988	15523	16934	15523
query68	6559	652	642	642
query69	583	362	341	341
query70	1626	1479	1416	1416
query71	405	302	322	302
query72	6520	3541	3497	3497
query73	734	330	318	318
query74	6371	5825	5832	5825
query75	4640	3660	3662	3660
query76	4233	1088	1106	1088
query77	546	257	250	250
query78	12677	11569	11859	11569
query79	7730	652	652	652
query80	2500	410	416	410
query81	532	234	231	231
query82	1558	102	96	96
query83	194	138	135	135
query84	256	70	70	70
query85	1426	337	359	337
query86	365	287	298	287
query87	3284	3065	3042	3042
query88	5204	2408	2429	2408
query89	346	289	277	277
query90	1752	211	211	211
query91	183	139	142	139
query92	60	54	51	51
query93	3265	560	563	560
query94	895	211	212	211
query95	1132	1051	1067	1051
query96	655	334	335	334
query97	6423	6346	6313	6313
query98	195	183	171	171
query99	3023	1076	829	829
Total cold run time: 309260 ms
Total hot run time: 203516 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.94 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 5ac5c02c299d3289b0b9a92c015708349377cc19, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.03
query3	0.26	0.05	0.05
query4	1.77	0.06	0.07
query5	0.53	0.52	0.53
query6	1.24	0.63	0.62
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.48	0.48
query10	0.53	0.52	0.53
query11	0.12	0.08	0.09
query12	0.11	0.08	0.09
query13	0.62	0.61	0.61
query14	0.78	0.78	0.79
query15	0.77	0.76	0.76
query16	0.35	0.36	0.36
query17	1.03	1.02	0.98
query18	0.21	0.25	0.27
query19	1.93	1.85	1.84
query20	0.01	0.01	0.02
query21	15.44	0.57	0.53
query22	2.03	2.39	1.73
query23	16.97	0.96	0.98
query24	4.56	3.92	2.00
query25	0.36	0.12	0.06
query26	0.84	0.16	0.18
query27	0.05	0.04	0.03
query28	4.82	0.75	0.77
query29	12.73	2.40	2.31
query30	0.57	0.58	0.55
query31	2.81	0.39	0.37
query32	3.39	0.50	0.50
query33	3.08	3.06	3.07
query34	15.23	4.79	4.78
query35	4.84	4.83	4.82
query36	1.09	1.01	1.02
query37	0.06	0.05	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.02
query40	0.16	0.14	0.15
query41	0.06	0.01	0.02
query42	0.02	0.02	0.01
query43	0.02	0.01	0.02
Total cold run time: 100.11 s
Total hot run time: 31.94 s

@justfortaste @w41ter
[bugfix](backup)(cooldown) cancel backup properly when be backup fail…
da71e93 
…ed (apache#38724)

Currently, when a backup job failed, but it still at the state of
SNAPSHOTING.

Cancel the cancel backup properly when be backup failed
@w41ter w41ter force-pushed the pick_38724_to_origin_branch-2.0 branch from 5ac5c02 to da71e93 Compare August 7, 2024 08:50
@w41ter
Copy link
Contributor Author

w41ter commented Aug 7, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Aug 7, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49888 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit da71e93fa5a7fcfdd7d85c1470850506f06259b6, data reload: false

------ Round 1 ----------------------------------
q1	17809	4349	4369	4349
q2	2054	161	149	149
q3	10463	1890	1900	1890
q4	10225	1295	1321	1295
q5	8450	3883	3961	3883
q6	235	128	134	128
q7	2046	1611	1574	1574
q8	9329	2730	2698	2698
q9	10635	10335	10354	10335
q10	8646	3567	3532	3532
q11	423	236	250	236
q12	470	317	305	305
q13	18408	3970	4039	3970
q14	349	326	338	326
q15	527	454	451	451
q16	697	582	571	571
q17	1124	963	975	963
q18	7275	6740	6867	6740
q19	1773	1688	1673	1673
q20	552	324	294	294
q21	4417	4142	4086	4086
q22	522	440	440	440
Total cold run time: 116429 ms
Total hot run time: 49888 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4313	4305	4307	4305
q2	321	241	234	234
q3	4157	4142	4126	4126
q4	2741	2742	2744	2742
q5	7111	7030	7064	7030
q6	238	123	118	118
q7	3228	2829	2881	2829
q8	4374	4539	4495	4495
q9	16762	16836	16755	16755
q10	4243	4242	4212	4212
q11	755	724	703	703
q12	1023	842	868	842
q13	6377	3761	3727	3727
q14	454	419	422	419
q15	511	455	462	455
q16	739	670	688	670
q17	3851	3862	3839	3839
q18	8801	8773	8915	8773
q19	1716	1711	1655	1655
q20	2385	2124	2140	2124
q21	8479	8495	8448	8448
q22	1056	991	1006	991
Total cold run time: 83635 ms
Total hot run time: 79492 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.74% (8102/21467)
Line Coverage: 29.37% (66350/225881)
Region Coverage: 28.88% (34239/118564)
Branch Coverage: 24.76% (17595/71052)
Coverage Report: http://coverage.selectdb-in.cc/coverage/da71e93fa5a7fcfdd7d85c1470850506f06259b6_da71e93fa5a7fcfdd7d85c1470850506f06259b6/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 203387 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit da71e93fa5a7fcfdd7d85c1470850506f06259b6, data reload: false

query1	944	383	404	383
query2	6528	2792	2716	2716
query3	6914	213	212	212
query4	20477	18069	17986	17986
query5	19735	6429	6545	6429
query6	281	219	238	219
query7	4153	294	317	294
query8	451	454	395	395
query9	3117	2707	2627	2627
query10	399	313	288	288
query11	11321	10701	10791	10701
query12	123	78	82	78
query13	5604	701	685	685
query14	17711	13504	13590	13504
query15	358	244	246	244
query16	6485	287	271	271
query17	1729	1466	870	870
query18	2318	423	411	411
query19	210	148	149	148
query20	80	79	78	78
query21	190	101	96	96
query22	5275	5121	5159	5121
query23	32749	31832	31915	31832
query24	7035	6593	6509	6509
query25	547	441	441	441
query26	534	162	164	162
query27	1883	300	293	293
query28	6177	2385	2354	2354
query29	2957	2853	2711	2711
query30	243	171	167	167
query31	905	745	758	745
query32	66	62	61	61
query33	405	250	261	250
query34	856	473	479	473
query35	1103	918	914	914
query36	1291	1213	1323	1213
query37	90	59	62	59
query38	3080	2938	2917	2917
query39	1365	1350	1326	1326
query40	204	93	97	93
query41	45	44	43	43
query42	86	85	84	84
query43	667	704	752	704
query44	1156	748	733	733
query45	248	236	233	233
query46	1221	951	937	937
query47	1673	1769	1727	1727
query48	1026	724	718	718
query49	625	372	369	369
query50	861	582	614	582
query51	4760	4637	4660	4637
query52	97	81	93	81
query53	446	317	320	317
query54	2671	2451	2446	2446
query55	89	75	81	75
query56	252	216	198	198
query57	1152	1059	1102	1059
query58	225	208	208	208
query59	4051	3749	4068	3749
query60	223	204	209	204
query61	99	93	99	93
query62	860	522	469	469
query63	477	339	334	334
query64	2596	1536	1388	1388
query65	3641	3572	3577	3572
query66	767	392	386	386
query67	15914	14932	15362	14932
query68	8935	639	668	639
query69	595	344	361	344
query70	1609	1399	1402	1399
query71	422	304	310	304
query72	6746	3519	3524	3519
query73	735	338	325	325
query74	6315	5922	5874	5874
query75	5418	3722	3643	3643
query76	5447	1133	1215	1133
query77	935	257	260	257
query78	12765	11543	11944	11543
query79	8652	645	656	645
query80	1322	403	406	403
query81	490	238	231	231
query82	1662	96	97	96
query83	170	135	135	135
query84	257	74	69	69
query85	880	328	338	328
query86	328	336	331	331
query87	3255	3036	3072	3036
query88	5116	2406	2425	2406
query89	378	299	291	291
query90	2006	216	211	211
query91	196	139	144	139
query92	57	58	53	53
query93	5577	581	590	581
query94	699	211	208	208
query95	1104	1068	1060	1060
query96	650	343	343	343
query97	6511	6356	6295	6295
query98	196	175	169	169
query99	2969	860	823	823
Total cold run time: 314218 ms
Total hot run time: 203387 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.57 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit da71e93fa5a7fcfdd7d85c1470850506f06259b6, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.03
query3	0.25	0.04	0.04
query4	1.79	0.06	0.06
query5	0.53	0.52	0.52
query6	1.26	0.60	0.62
query7	0.01	0.01	0.02
query8	0.03	0.02	0.02
query9	0.53	0.49	0.49
query10	0.54	0.52	0.53
query11	0.11	0.08	0.08
query12	0.12	0.09	0.09
query13	0.61	0.61	0.60
query14	0.76	0.79	0.81
query15	0.76	0.76	0.75
query16	0.36	0.37	0.37
query17	1.00	1.01	1.02
query18	0.21	0.27	0.24
query19	1.87	1.86	1.85
query20	0.01	0.02	0.01
query21	15.45	0.54	0.58
query22	1.85	2.26	1.52
query23	17.14	1.00	0.95
query24	4.67	0.95	1.80
query25	0.33	0.14	0.04
query26	0.59	0.17	0.15
query27	0.04	0.04	0.04
query28	7.94	0.74	0.73
query29	12.62	2.25	2.24
query30	0.59	0.58	0.53
query31	2.80	0.38	0.37
query32	3.39	0.50	0.49
query33	3.08	3.08	3.08
query34	15.24	4.77	4.79
query35	4.84	4.82	4.86
query36	1.04	1.02	1.01
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.02
query43	0.03	0.01	0.01
Total cold run time: 102.84 s
Total hot run time: 30.57 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit da71e93fa5a7fcfdd7d85c1470850506f06259b6 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.1 seconds inserted 10000000 Rows, about 473K ops/s

@w41ter w41ter merged commit 8528a1b into apache:branch-2.0 Aug 7, 2024
21 of 24 checks passed
@w41ter w41ter deleted the pick_38724_to_origin_branch-2.0 branch August 7, 2024 11:44
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
@w41ter @justfortaste
[bugfix](backup)(cooldown) cancel backup properly when be backup failed
6c95de9 
apache#38724 (apache#38994)

cherry pick from apache#38724

Co-authored-by: zhangyuan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
kind/test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@w41ter @doris-robot @justfortaste