Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](function) Fix wrong result of week/yearweek in constant folding #37376

Merged
merged 1 commit into from
Jul 8, 2024

Conversation

zclllyybb
Copy link
Contributor

@zclllyybb zclllyybb commented Jul 6, 2024

Proposed changes

Issue Number: close #xxx

Java doesn't support the calculation of 0000-01-01(0000 year is 1 B.C. which ISO-8601 defines). so some of our calc was wrong in FE. now fixed them.

new result:

mysql> select week('0000-01-01', 0),week('0000-01-01', 1),week('0000-01-01', 2),week('0000-01-01', 3),week('0000-01-01', 4), week('0000-01-01', 5), week('0000-01-01', 6), week('0000-01-01', 7);
+------+------+------+------+------+------+------+------+
| 1    | 0    | 1    | 52   | 1    | 0    | 1    | 52   |
+------+------+------+------+------+------+------+------+
|    1 |    0 |    1 |   52 |    1 |    0 |    1 |   52 |
+------+------+------+------+------+------+------+------+
1 row in set (0.10 sec)

mysql> select week('0000-01-02', 0),week('0000-01-02', 1),week('0000-01-02', 2),week('0000-01-02', 3),week('0000-01-02', 4), week('0000-01-02', 5), week('0000-01-02', 6), week('0000-01-02', 7);
+------+------+------+------+------+------+------+------+
| 1    | 1    | 1    | 1    | 1    | 1    | 1    | 1    |
+------+------+------+------+------+------+------+------+
|    1 |    1 |    1 |    1 |    1 |    1 |    1 |    1 |
+------+------+------+------+------+------+------+------+
1 row in set (0.10 sec)

same with yearweek

@zclllyybb
Copy link
Contributor Author

run buildall

@zclllyybb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39684 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 167ceee02fffbf2103caceab575127b8f908254e, data reload: false

------ Round 1 ----------------------------------
q1	17618	4796	4274	4274
q2	2024	201	189	189
q3	10434	1212	1011	1011
q4	10199	829	785	785
q5	7483	2621	2655	2621
q6	220	136	137	136
q7	946	602	607	602
q8	9222	2075	2033	2033
q9	8736	6497	6486	6486
q10	8964	3757	3710	3710
q11	456	241	246	241
q12	492	236	223	223
q13	18924	2944	2988	2944
q14	253	238	233	233
q15	556	499	492	492
q16	529	377	377	377
q17	960	642	687	642
q18	7931	7447	7417	7417
q19	5827	1529	1384	1384
q20	653	335	336	335
q21	4847	3219	3213	3213
q22	397	336	336	336
Total cold run time: 117671 ms
Total hot run time: 39684 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4358	4270	4198	4198
q2	369	262	262	262
q3	2965	2892	2823	2823
q4	1993	1641	1648	1641
q5	5651	5511	5447	5447
q6	226	132	133	132
q7	2219	1881	1846	1846
q8	3306	3405	3416	3405
q9	8606	8762	8983	8762
q10	4148	3963	3772	3772
q11	612	529	497	497
q12	858	645	644	644
q13	16069	3206	3205	3205
q14	318	279	291	279
q15	534	488	479	479
q16	498	435	446	435
q17	1822	1542	1552	1542
q18	8093	7839	7815	7815
q19	1799	1650	1610	1610
q20	2178	1862	1886	1862
q21	5179	5082	4705	4705
q22	628	586	574	574
Total cold run time: 72429 ms
Total hot run time: 55935 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173720 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 167ceee02fffbf2103caceab575127b8f908254e, data reload: false

query1	903	381	370	370
query2	6464	2526	2410	2410
query3	6639	206	216	206
query4	28494	17425	17247	17247
query5	3642	474	471	471
query6	259	170	164	164
query7	4593	310	303	303
query8	313	310	309	309
query9	8481	2375	2346	2346
query10	577	299	312	299
query11	10674	10103	10077	10077
query12	117	86	87	86
query13	1641	389	388	388
query14	10199	6976	7566	6976
query15	231	186	183	183
query16	7933	308	302	302
query17	1835	542	514	514
query18	2064	274	275	274
query19	218	151	150	150
query20	88	82	84	82
query21	220	122	133	122
query22	4221	4099	4058	4058
query23	34279	33708	33640	33640
query24	10578	2887	2835	2835
query25	574	384	391	384
query26	698	159	155	155
query27	2252	328	330	328
query28	5797	2152	2131	2131
query29	882	673	611	611
query30	265	166	156	156
query31	949	733	772	733
query32	94	53	60	53
query33	697	297	301	297
query34	907	506	478	478
query35	734	654	648	648
query36	1160	978	979	978
query37	136	78	81	78
query38	2991	2842	2812	2812
query39	929	861	836	836
query40	207	137	131	131
query41	53	55	54	54
query42	110	101	104	101
query43	602	563	565	563
query44	1129	736	746	736
query45	195	160	160	160
query46	1067	738	722	722
query47	1854	1806	1779	1779
query48	379	306	297	297
query49	870	416	432	416
query50	767	386	398	386
query51	7001	6861	6788	6788
query52	98	101	93	93
query53	378	301	300	300
query54	888	444	446	444
query55	73	76	74	74
query56	283	271	269	269
query57	1102	1049	1069	1049
query58	254	241	252	241
query59	3603	3464	3276	3276
query60	297	274	276	274
query61	94	94	95	94
query62	629	434	443	434
query63	321	291	287	287
query64	9127	2165	1641	1641
query65	3156	3110	3116	3110
query66	747	319	334	319
query67	15327	15105	15199	15105
query68	4722	525	532	525
query69	628	474	372	372
query70	1189	1110	1069	1069
query71	439	285	284	284
query72	7561	5884	5552	5552
query73	762	323	324	323
query74	5972	5519	5441	5441
query75	3667	2652	2676	2652
query76	3337	922	913	913
query77	597	297	307	297
query78	9627	8837	9571	8837
query79	2206	511	514	511
query80	2275	470	532	470
query81	598	221	222	221
query82	1021	107	104	104
query83	298	169	174	169
query84	265	95	90	90
query85	1362	318	297	297
query86	468	302	308	302
query87	3257	3121	3100	3100
query88	3840	2445	2465	2445
query89	489	382	374	374
query90	1771	190	193	190
query91	134	103	106	103
query92	60	51	49	49
query93	2579	509	505	505
query94	1076	212	211	211
query95	417	318	321	318
query96	604	265	269	265
query97	3186	3048	3054	3048
query98	225	194	195	194
query99	1274	856	864	856
Total cold run time: 280245 ms
Total hot run time: 173720 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.61 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 167ceee02fffbf2103caceab575127b8f908254e, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.03	0.04
query3	0.23	0.04	0.04
query4	1.69	0.07	0.07
query5	0.50	0.48	0.49
query6	1.13	0.73	0.73
query7	0.02	0.01	0.01
query8	0.05	0.04	0.04
query9	0.55	0.49	0.50
query10	0.55	0.56	0.54
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.60	0.59	0.58
query14	0.76	0.77	0.79
query15	0.84	0.81	0.81
query16	0.34	0.37	0.36
query17	0.95	1.01	0.97
query18	0.25	0.25	0.23
query19	1.79	1.72	1.71
query20	0.02	0.01	0.01
query21	15.40	0.78	0.66
query22	4.44	7.27	2.03
query23	18.31	1.43	1.24
query24	2.13	0.24	0.21
query25	0.15	0.08	0.08
query26	0.29	0.21	0.20
query27	0.46	0.24	0.22
query28	13.29	1.01	1.00
query29	12.77	3.33	3.32
query30	0.26	0.06	0.06
query31	2.87	0.39	0.38
query32	3.27	0.49	0.46
query33	2.93	2.86	2.91
query34	16.97	4.34	4.43
query35	4.44	4.39	4.41
query36	0.65	0.46	0.48
query37	0.19	0.16	0.16
query38	0.16	0.14	0.14
query39	0.05	0.03	0.03
query40	0.16	0.12	0.12
query41	0.09	0.04	0.04
query42	0.05	0.04	0.04
query43	0.04	0.04	0.04
Total cold run time: 110.06 s
Total hot run time: 30.61 s

Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Jul 8, 2024
Copy link
Contributor

github-actions bot commented Jul 8, 2024

PR approved by anyone and no changes requested.

@zhangstar333 zhangstar333 merged commit 0c5703f into apache:master Jul 8, 2024
26 of 29 checks passed
dataroaring pushed a commit that referenced this pull request Jul 17, 2024
…#37376)

## Proposed changes

Issue Number: close #xxx

Java doesn't support the calculation of `0000-01-01`(0000 year is 1 B.C.
which ISO-8601 defines). so some of our calc was wrong in FE. now fixed
them.

new result:
```sql
mysql> select week('0000-01-01', 0),week('0000-01-01', 1),week('0000-01-01', 2),week('0000-01-01', 3),week('0000-01-01', 4), week('0000-01-01', 5), week('0000-01-01', 6), week('0000-01-01', 7);
+------+------+------+------+------+------+------+------+
| 1    | 0    | 1    | 52   | 1    | 0    | 1    | 52   |
+------+------+------+------+------+------+------+------+
|    1 |    0 |    1 |   52 |    1 |    0 |    1 |   52 |
+------+------+------+------+------+------+------+------+
1 row in set (0.10 sec)

mysql> select week('0000-01-02', 0),week('0000-01-02', 1),week('0000-01-02', 2),week('0000-01-02', 3),week('0000-01-02', 4), week('0000-01-02', 5), week('0000-01-02', 6), week('0000-01-02', 7);
+------+------+------+------+------+------+------+------+
| 1    | 1    | 1    | 1    | 1    | 1    | 1    | 1    |
+------+------+------+------+------+------+------+------+
|    1 |    1 |    1 |    1 |    1 |    1 |    1 |    1 |
+------+------+------+------+------+------+------+------+
1 row in set (0.10 sec)
```
same with `yearweek`
@zclllyybb zclllyybb deleted the week branch February 7, 2025 15:37
zclllyybb added a commit to zclllyybb/doris that referenced this pull request Feb 7, 2025
…apache#37376)

Issue Number: close #xxx

Java doesn't support the calculation of `0000-01-01`(0000 year is 1 B.C.
which ISO-8601 defines). so some of our calc was wrong in FE. now fixed
them.

new result:
```sql
mysql> select week('0000-01-01', 0),week('0000-01-01', 1),week('0000-01-01', 2),week('0000-01-01', 3),week('0000-01-01', 4), week('0000-01-01', 5), week('0000-01-01', 6), week('0000-01-01', 7);
+------+------+------+------+------+------+------+------+
| 1    | 0    | 1    | 52   | 1    | 0    | 1    | 52   |
+------+------+------+------+------+------+------+------+
|    1 |    0 |    1 |   52 |    1 |    0 |    1 |   52 |
+------+------+------+------+------+------+------+------+
1 row in set (0.10 sec)

mysql> select week('0000-01-02', 0),week('0000-01-02', 1),week('0000-01-02', 2),week('0000-01-02', 3),week('0000-01-02', 4), week('0000-01-02', 5), week('0000-01-02', 6), week('0000-01-02', 7);
+------+------+------+------+------+------+------+------+
| 1    | 1    | 1    | 1    | 1    | 1    | 1    | 1    |
+------+------+------+------+------+------+------+------+
|    1 |    1 |    1 |    1 |    1 |    1 |    1 |    1 |
+------+------+------+------+------+------+------+------+
1 row in set (0.10 sec)
```
same with `yearweek`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.1-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants