[enhance](cloud) Add a policy when be abnormal, tablet delay switch be

[deardeng]

No description provided.

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[deardeng]

run buildall

[doris-robot]

TPC-H: Total hot run time: 38015 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2b0075b9affa992b7f04eb8f40df88ab4f55ed8, data reload: false

------ Round 1 ----------------------------------
q1	17612	4548	4366	4366
q2	2031	191	185	185
q3	11713	933	1084	933
q4	10466	797	722	722
q5	7721	2858	2806	2806
q6	226	139	140	139
q7	968	631	605	605
q8	9314	2059	2090	2059
q9	6954	6555	6536	6536
q10	7012	2274	2253	2253
q11	461	249	255	249
q12	394	220	224	220
q13	18933	3066	3077	3066
q14	272	247	255	247
q15	529	489	489	489
q16	530	442	435	435
q17	985	737	760	737
q18	7352	6907	6888	6888
q19	1405	976	949	949
q20	689	358	334	334
q21	4017	3040	2789	2789
q22	1117	1008	1031	1008
Total cold run time: 110701 ms
Total hot run time: 38015 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4447	4343	4461	4343
q2	382	280	279	279
q3	2909	2727	2654	2654
q4	1905	1689	1655	1655
q5	5752	5721	5783	5721
q6	242	148	144	144
q7	2243	1836	1852	1836
q8	3332	3502	3507	3502
q9	8957	8868	8989	8868
q10	3674	3511	3414	3414
q11	620	525	536	525
q12	870	693	716	693
q13	14411	3360	3487	3360
q14	343	303	306	303
q15	554	509	509	509
q16	582	541	501	501
q17	1858	1607	1546	1546
q18	8335	7887	8040	7887
q19	1812	1457	1548	1457
q20	2190	1903	1947	1903
q21	5674	5500	5498	5498
q22	1198	1112	1030	1030
Total cold run time: 72290 ms
Total hot run time: 57628 ms

[doris-robot]

TPC-DS: Total hot run time: 192006 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a2b0075b9affa992b7f04eb8f40df88ab4f55ed8, data reload: false

query1	1256	902	922	902
query2	6315	1859	1942	1859
query3	10601	4002	3704	3704
query4	59454	23849	23203	23203
query5	5503	500	480	480
query6	416	177	158	158
query7	5953	301	281	281
query8	313	217	223	217
query9	9121	2479	2480	2479
query10	508	260	275	260
query11	18192	15089	15224	15089
query12	156	102	107	102
query13	1624	372	378	372
query14	11205	7266	7739	7266
query15	245	184	175	175
query16	7510	465	465	465
query17	1140	565	560	560
query18	2064	309	296	296
query19	283	149	154	149
query20	129	121	112	112
query21	209	113	106	106
query22	4832	4359	4363	4359
query23	34175	33355	33366	33355
query24	5909	2839	2823	2823
query25	564	408	399	399
query26	692	160	158	158
query27	1782	280	279	279
query28	3603	2038	2007	2007
query29	712	433	425	425
query30	234	157	154	154
query31	925	763	764	763
query32	90	54	57	54
query33	417	296	289	289
query34	870	478	465	465
query35	837	719	730	719
query36	1088	930	937	930
query37	151	87	84	84
query38	3982	3867	3890	3867
query39	1455	1397	1404	1397
query40	194	119	117	117
query41	49	50	44	44
query42	122	96	95	95
query43	541	475	469	469
query44	1082	756	745	745
query45	198	168	167	167
query46	1105	799	757	757
query47	1916	1788	1820	1788
query48	371	294	301	294
query49	778	447	456	447
query50	819	412	423	412
query51	7120	6813	6854	6813
query52	98	89	88	88
query53	252	179	181	179
query54	568	482	465	465
query55	81	73	78	73
query56	303	298	262	262
query57	1182	1093	1079	1079
query58	228	234	230	230
query59	3055	2813	2727	2727
query60	296	281	287	281
query61	151	141	204	141
query62	743	673	623	623
query63	222	179	184	179
query64	2798	709	655	655
query65	3193	3109	3171	3109
query66	636	330	345	330
query67	15286	15295	15240	15240
query68	4387	567	562	562
query69	414	276	270	270
query70	1189	1087	1043	1043
query71	354	279	276	276
query72	6559	4023	4033	4023
query73	742	323	324	323
query74	9262	8826	8789	8789
query75	3359	2689	2714	2689
query76	1668	1062	1062	1062
query77	570	323	317	317
query78	9739	8967	9869	8967
query79	1034	533	519	519
query80	921	520	528	520
query81	565	232	226	226
query82	241	142	140	140
query83	225	147	150	147
query84	265	75	74	74
query85	974	282	282	282
query86	372	304	292	292
query87	4424	4222	4160	4160
query88	3285	2285	2274	2274
query89	385	289	283	283
query90	1907	188	188	188
query91	119	99	101	99
query92	57	49	48	48
query93	1074	529	535	529
query94	798	297	300	297
query95	342	248	253	248
query96	583	259	260	259
query97	3213	3068	3101	3068
query98	211	199	189	189
query99	1636	1272	1290	1272
Total cold run time: 308776 ms
Total hot run time: 192006 ms

[doris-robot]

ClickBench: Total hot run time: 31.2 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a2b0075b9affa992b7f04eb8f40df88ab4f55ed8, data reload: false

query1	0.05	0.05	0.04
query2	0.07	0.03	0.04
query3	0.23	0.05	0.05
query4	1.69	0.08	0.07
query5	0.51	0.50	0.49
query6	1.14	0.73	0.72
query7	0.02	0.02	0.02
query8	0.05	0.04	0.04
query9	0.54	0.49	0.48
query10	0.54	0.56	0.55
query11	0.15	0.12	0.12
query12	0.15	0.13	0.12
query13	0.60	0.58	0.59
query14	1.42	1.43	1.47
query15	0.84	0.82	0.80
query16	0.36	0.36	0.37
query17	1.03	1.08	0.96
query18	0.20	0.20	0.20
query19	1.84	1.78	1.78
query20	0.01	0.01	0.01
query21	15.43	0.69	0.68
query22	4.19	7.39	1.79
query23	18.28	1.37	1.27
query24	2.20	0.22	0.23
query25	0.15	0.09	0.08
query26	0.26	0.18	0.17
query27	0.08	0.08	0.07
query28	13.17	1.01	1.00
query29	12.63	3.31	3.29
query30	0.24	0.06	0.05
query31	2.89	0.40	0.38
query32	3.26	0.47	0.48
query33	3.01	3.01	3.00
query34	16.99	4.38	4.44
query35	4.43	4.44	4.48
query36	0.66	0.47	0.47
query37	0.18	0.15	0.15
query38	0.16	0.15	0.14
query39	0.05	0.04	0.04
query40	0.16	0.13	0.12
query41	0.09	0.05	0.05
query42	0.06	0.06	0.05
query43	0.04	0.04	0.05
Total cold run time: 110.05 s
Total hot run time: 31.2 s

[github-actions]

PR approved by at least one committer and no changes requested.

[github-actions]

PR approved by anyone and no changes requested.

[yujun777]

LGTM