[fix](routine load) add read lock to fix some concurrent bugs (#39242)

[sollhui]

pick #39242

[doris-robot]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

[sollhui]

run buildall

[doris-robot]

TPC-H: Total hot run time: 49195 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9f0920736252ac18735888627bfbc816902213a, data reload: false

------ Round 1 ----------------------------------
q1	17681	4465	4395	4395
q2	2059	188	152	152
q3	10328	1934	1931	1931
q4	10407	1267	1315	1267
q5	8462	3847	3943	3847
q6	228	120	121	120
q7	2016	1641	1586	1586
q8	9295	2734	2703	2703
q9	10322	9852	9861	9852
q10	8627	3541	3522	3522
q11	411	238	238	238
q12	458	295	297	295
q13	18354	3981	4044	3981
q14	364	329	330	329
q15	511	448	456	448
q16	552	449	449	449
q17	1132	961	930	930
q18	7179	6910	6889	6889
q19	1683	1564	1502	1502
q20	513	312	302	302
q21	4374	4139	4067	4067
q22	476	398	390	390
Total cold run time: 115432 ms
Total hot run time: 49195 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4282	4305	4268	4268
q2	317	224	219	219
q3	4172	4152	4126	4126
q4	2740	2736	2744	2736
q5	7234	7132	7183	7132
q6	232	118	118	118
q7	3255	2809	2855	2809
q8	4359	4458	4498	4458
q9	13873	13503	14157	13503
q10	4214	4278	4241	4241
q11	725	683	694	683
q12	1033	875	876	875
q13	6965	3724	3737	3724
q14	455	431	430	430
q15	514	472	460	460
q16	658	609	600	600
q17	3888	3865	3854	3854
q18	8723	8783	8774	8774
q19	1718	1648	1644	1644
q20	2370	2105	2104	2104
q21	8599	8491	8348	8348
q22	1019	960	947	947
Total cold run time: 81345 ms
Total hot run time: 76053 ms

[doris-robot]

TPC-DS: Total hot run time: 212832 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9f0920736252ac18735888627bfbc816902213a, data reload: false

query1	930	395	422	395
query2	6563	2114	2145	2114
query3	6926	208	207	207
query4	23510	21450	21710	21450
query5	19728	6488	6541	6488
query6	289	233	239	233
query7	4335	306	326	306
query8	280	255	236	236
query9	3082	2659	2613	2613
query10	488	306	307	306
query11	15580	15106	14949	14949
query12	131	76	73	73
query13	1038	437	452	437
query14	17110	13834	14065	13834
query15	363	216	221	216
query16	6187	273	260	260
query17	1775	929	908	908
query18	895	321	320	320
query19	211	154	163	154
query20	81	74	81	74
query21	195	98	97	97
query22	5048	5092	4998	4998
query23	34352	33481	33445	33445
query24	7819	6275	6419	6275
query25	538	434	447	434
query26	1305	167	168	167
query27	2391	299	303	299
query28	6077	2296	2251	2251
query29	2867	2745	2639	2639
query30	248	165	169	165
query31	974	747	756	747
query32	68	62	61	61
query33	457	276	267	267
query34	890	494	491	491
query35	1128	901	902	901
query36	1402	1311	1244	1244
query37	167	60	61	60
query38	3095	2959	2959	2959
query39	1389	1336	1304	1304
query40	316	91	99	91
query41	41	37	37	37
query42	88	95	84	84
query43	634	617	569	569
query44	1174	716	718	716
query45	245	232	233	232
query46	1245	957	964	957
query47	1852	1834	1759	1759
query48	511	417	405	405
query49	656	381	378	378
query50	861	624	577	577
query51	5367	4760	4631	4631
query52	97	79	81	79
query53	232	189	182	182
query54	2680	2485	2483	2483
query55	88	85	87	85
query56	255	221	220	220
query57	1192	1096	1102	1096
query58	219	207	213	207
query59	3665	3459	3311	3311
query60	229	208	215	208
query61	97	95	96	95
query62	854	461	463	461
query63	210	180	177	177
query64	3549	1578	1467	1467
query65	3634	3575	3558	3558
query66	777	432	411	411
query67	17608	15241	15134	15134
query68	9660	657	662	657
query69	511	283	280	280
query70	1562	1449	1402	1402
query71	412	311	304	304
query72	6791	4810	4774	4774
query73	769	321	325	321
query74	6297	5847	5894	5847
query75	5228	3732	3710	3710
query76	5323	1128	1219	1128
query77	938	261	262	261
query78	12889	11987	11582	11582
query79	5296	643	641	641
query80	821	389	385	385
query81	487	241	240	240
query82	1338	103	107	103
query83	180	138	140	138
query84	260	76	69	69
query85	802	320	317	317
query86	324	279	293	279
query87	3236	3055	3045	3045
query88	4686	2321	2308	2308
query89	375	285	289	285
query90	1971	211	222	211
query91	159	127	130	127
query92	61	53	53	53
query93	4341	581	553	553
query94	658	207	211	207
query95	2014	1958	1968	1958
query96	642	320	334	320
query97	6559	6290	6423	6290
query98	222	213	226	213
query99	2657	859	837	837
Total cold run time: 315757 ms
Total hot run time: 212832 ms

[doris-robot]

ClickBench: Total hot run time: 31.15 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9f0920736252ac18735888627bfbc816902213a, data reload: false

query1	0.02	0.02	0.03
query2	0.07	0.03	0.03
query3	0.26	0.05	0.05
query4	1.79	0.06	0.06
query5	0.53	0.53	0.52
query6	1.22	0.61	0.62
query7	0.01	0.01	0.01
query8	0.04	0.03	0.02
query9	0.53	0.49	0.48
query10	0.54	0.53	0.54
query11	0.13	0.08	0.08
query12	0.12	0.09	0.10
query13	0.62	0.62	0.61
query14	0.78	0.79	0.81
query15	0.79	0.76	0.76
query16	0.38	0.38	0.38
query17	0.97	1.05	0.97
query18	0.21	0.27	0.25
query19	1.85	1.89	1.89
query20	0.01	0.01	0.01
query21	15.46	0.55	0.56
query22	2.30	2.47	2.30
query23	17.44	1.07	0.86
query24	6.47	0.67	1.42
query25	0.40	0.14	0.05
query26	0.64	0.16	0.15
query27	0.04	0.04	0.03
query28	6.62	0.74	0.75
query29	12.62	2.38	2.32
query30	0.58	0.52	0.53
query31	2.82	0.39	0.38
query32	3.38	0.49	0.52
query33	3.07	3.12	3.06
query34	15.25	4.79	4.80
query35	4.84	4.81	4.84
query36	1.05	1.02	1.02
query37	0.06	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.06	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.02	0.02
Total cold run time: 104.22 s
Total hot run time: 31.15 s

[doris-robot]

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c9f0920736252ac18735888627bfbc816902213a with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s