Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](binlog) Fix NPE when recover binlogs #39909

Merged
merged 1 commit into from
Aug 26, 2024
Merged

[fix](binlog) Fix NPE when recover binlogs #39909

merged 1 commit into from
Aug 26, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Aug 26, 2024

The field partition ID of DropPartitionInfo was added in PR: #37196, the old version doesn't contain this field so it will be null.

@w41ter
[fix](binlog) Fix NPE when recover binlogs
00033eb 
The field partition ID of DropPartitionInfo was added in PR: apache#37196,
the old version doesn't contain this field so it will be null.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented Aug 26, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

w41ter added a commit to w41ter/incubator-doris that referenced this pull request Aug 26, 2024
@w41ter
[fix](binlog) Fix NPE when recover binlogs (apache#39909)
a35f7a7 
The field partition ID of DropPartitionInfo was added in PR: apache#37196,
the old version doesn't contain this field so it will be null.
@w41ter w41ter mentioned this pull request Aug 26, 2024
Merged
w41ter added a commit to w41ter/incubator-doris that referenced this pull request Aug 26, 2024
@w41ter
[fix](binlog) Fix NPE when recover binlogs (apache#39909)
4d6209c 
The field partition ID of DropPartitionInfo was added in PR: apache#37196,
the old version doesn't contain this field so it will be null.
@doris-robot
Copy link

TPC-H: Total hot run time: 38089 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 00033eb1bd022154727c5e968828220cc7c9cbef, data reload: false

------ Round 1 ----------------------------------
q1	17610	4327	4294	4294
q2	2017	186	176	176
q3	11809	954	1144	954
q4	10502	724	714	714
q5	7747	2867	2870	2867
q6	224	136	137	136
q7	981	628	617	617
q8	9355	2060	2097	2060
q9	7163	6524	6509	6509
q10	6985	2229	2211	2211
q11	461	247	254	247
q12	393	229	230	229
q13	17992	3087	3035	3035
q14	278	232	239	232
q15	515	491	487	487
q16	504	405	389	389
q17	991	688	775	688
q18	7317	6854	6917	6854
q19	1398	1079	1002	1002
q20	692	336	342	336
q21	4185	3074	3042	3042
q22	1103	1010	1016	1010
Total cold run time: 110222 ms
Total hot run time: 38089 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4363	4314	4257	4257
q2	376	281	272	272
q3	2908	2709	2717	2709
q4	1960	1621	1666	1621
q5	5633	5724	5765	5724
q6	235	142	142	142
q7	2246	1933	1855	1855
q8	3257	3544	3506	3506
q9	8919	8907	8847	8847
q10	3641	3430	3354	3354
q11	599	516	521	516
q12	856	709	660	660
q13	14629	3135	3237	3135
q14	330	295	316	295
q15	533	503	498	498
q16	505	444	446	444
q17	1838	1552	1572	1552
q18	8169	7757	7804	7757
q19	1751	1554	1493	1493
q20	2167	1963	1870	1870
q21	5799	5360	5762	5360
q22	1134	1021	1041	1021
Total cold run time: 71848 ms
Total hot run time: 56888 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192456 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 00033eb1bd022154727c5e968828220cc7c9cbef, data reload: false

query1	1261	898	867	867
query2	6339	1936	1941	1936
query3	10596	3936	4013	3936
query4	60151	25810	23411	23411
query5	5444	498	485	485
query6	404	161	168	161
query7	5766	303	307	303
query8	283	202	212	202
query9	8856	2497	2465	2465
query10	500	270	262	262
query11	18457	14971	15418	14971
query12	172	102	101	101
query13	1542	396	395	395
query14	11012	7360	7448	7360
query15	232	170	168	168
query16	7532	463	507	463
query17	1115	569	584	569
query18	1978	287	297	287
query19	282	151	142	142
query20	116	111	110	110
query21	208	103	106	103
query22	4653	4582	4666	4582
query23	34527	33416	33250	33250
query24	5941	2887	2884	2884
query25	527	374	371	371
query26	682	160	158	158
query27	1789	283	282	282
query28	3738	2049	2022	2022
query29	695	402	409	402
query30	233	149	150	149
query31	937	781	753	753
query32	84	56	57	56
query33	451	285	280	280
query34	857	493	477	477
query35	859	727	700	700
query36	1100	954	946	946
query37	145	83	85	83
query38	3957	3852	3895	3852
query39	1439	1437	1412	1412
query40	206	118	121	118
query41	50	47	45	45
query42	119	103	103	103
query43	515	468	459	459
query44	1096	742	764	742
query45	201	165	170	165
query46	1096	746	745	745
query47	1871	1805	1822	1805
query48	370	301	296	296
query49	788	448	461	448
query50	839	437	440	437
query51	7275	7211	7169	7169
query52	101	92	89	89
query53	256	187	189	187
query54	577	469	466	466
query55	81	79	78	78
query56	304	269	271	269
query57	1223	1081	1063	1063
query58	228	238	272	238
query59	2898	2867	2631	2631
query60	304	279	283	279
query61	129	127	124	124
query62	759	643	664	643
query63	214	189	182	182
query64	3391	1872	1869	1869
query65	3221	3160	3182	3160
query66	655	346	348	346
query67	15417	15086	15384	15086
query68	3431	574	581	574
query69	422	294	296	294
query70	1130	1129	1081	1081
query71	375	277	274	274
query72	5871	2464	2233	2233
query73	759	324	321	321
query74	9383	8723	8887	8723
query75	3395	2727	2728	2727
query76	1489	1031	1013	1013
query77	544	330	332	330
query78	9810	9154	9052	9052
query79	1592	533	526	526
query80	822	506	513	506
query81	561	231	230	230
query82	272	139	138	138
query83	181	148	150	148
query84	272	74	78	74
query85	910	314	278	278
query86	406	304	300	300
query87	4452	4164	4339	4164
query88	4126	2356	2328	2328
query89	399	289	286	286
query90	2032	199	199	199
query91	123	100	99	99
query92	61	54	52	52
query93	1917	539	544	539
query94	830	301	288	288
query95	350	271	268	268
query96	616	275	276	275
query97	3237	3087	3042	3042
query98	221	201	196	196
query99	1723	1302	1300	1300
Total cold run time: 310756 ms
Total hot run time: 192456 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.9 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 00033eb1bd022154727c5e968828220cc7c9cbef, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.23	0.04	0.05
query4	1.67	0.10	0.08
query5	0.50	0.51	0.48
query6	1.12	0.73	0.73
query7	0.02	0.01	0.01
query8	0.06	0.05	0.05
query9	0.54	0.50	0.47
query10	0.55	0.54	0.54
query11	0.16	0.12	0.11
query12	0.14	0.12	0.12
query13	0.63	0.58	0.58
query14	0.76	0.80	0.78
query15	0.84	0.83	0.83
query16	0.37	0.37	0.37
query17	1.02	1.06	1.02
query18	0.21	0.20	0.21
query19	1.99	1.76	1.84
query20	0.01	0.01	0.02
query21	15.41	0.67	0.66
query22	4.21	6.51	2.16
query23	18.32	1.36	1.29
query24	2.07	0.22	0.24
query25	0.14	0.08	0.08
query26	0.27	0.18	0.18
query27	0.07	0.08	0.08
query28	13.25	1.02	1.00
query29	12.63	3.37	3.28
query30	0.24	0.06	0.06
query31	2.88	0.41	0.39
query32	3.26	0.48	0.47
query33	2.96	3.03	2.97
query34	17.05	4.34	4.31
query35	4.40	4.40	4.45
query36	0.65	0.48	0.49
query37	0.19	0.16	0.16
query38	0.16	0.15	0.15
query39	0.05	0.04	0.03
query40	0.16	0.14	0.13
query41	0.10	0.05	0.05
query42	0.05	0.05	0.05
query43	0.05	0.04	0.04
Total cold run time: 109.52 s
Total hot run time: 30.9 s

dataroaring
dataroaring approved these changes Aug 26, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 26, 2024
@dataroaring dataroaring merged commit 5eb4824 into apache:master Aug 26, 2024
29 of 31 checks passed
@w41ter w41ter deleted the fix_drop_partition_info_comp branch August 26, 2024 11:41
@w41ter w41ter mentioned this pull request Aug 26, 2024
Merged
w41ter added a commit that referenced this pull request Aug 26, 2024
@w41ter
[fix](binlog) Fix NPE when recover binlogs (#39909) (#39911)
e7435bd 
Cherry-pick #39909 

The field partition ID of DropPartitionInfo was added in PR:
#37196, the old version doesn't contain this field so it
will be null.
w41ter added a commit that referenced this pull request Aug 26, 2024
@w41ter
[fix](binlog) Fix NPE when recover binlogs (#39909) (#39910)
fb5002d 
Cherry-pick #39909 

The field partition ID of DropPartitionInfo was added in PR:
#37196, the old version doesn't contain this field so it
will be null.
dataroaring pushed a commit that referenced this pull request Aug 26, 2024
@w41ter @dataroaring
[fix](binlog) Fix NPE when recover binlogs (#39909)
ccd6809 
The field partition ID of DropPartitionInfo was added in PR:
#37196, the old version doesn't contain this field so it
will be null.
@gavinchou gavinchou mentioned this pull request Oct 13, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yukang-Lian Yukang-Lian Yukang-Lian approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.15-merged dev/2.1.6-merged dev/3.0.2-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@w41ter @doris-robot @Yukang-Lian @dataroaring