Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: async linting #4460

Merged
merged 5 commits into from
Dec 2, 2024
Merged

feat: async linting #4460

merged 5 commits into from
Dec 2, 2024

Conversation

Kha
Copy link
Member

@Kha Kha commented Jun 14, 2024

This PR runs all linters for a single command (together) on a separate thread from further elaboration, making a first step towards parallelizing the elaborator.

@Kha
Copy link
Member Author

Kha commented Jun 14, 2024

!bench

@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Jun 14, 2024
@leanprover-community-mathlib4-bot
Copy link
Collaborator

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 237f392cc1a022ab710e388cc877e02cbcbfdd56 --onto bedcbfcfeed429428c3e7f008b6984fc8c2b2b76. (2024-06-14 14:48:40)

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit aa161a6.
The entire run failed.
Found no significant differences.

@Kha
Copy link
Member Author

Kha commented Jun 18, 2024

!bench

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit aa161a6.
The entire run failed.
Found no significant differences.

@Kha Kha force-pushed the parallel-linting branch from aa161a6 to 2873696 Compare July 5, 2024 15:44
leanprover-community-mathlib4-bot added a commit to leanprover-community/batteries that referenced this pull request Jul 5, 2024
leanprover-community-mathlib4-bot added a commit to leanprover-community/mathlib4 that referenced this pull request Jul 5, 2024
@Kha
Copy link
Member Author

Kha commented Jul 7, 2024

!bench

leanprover-community-mathlib4-bot added a commit to leanprover-community/batteries that referenced this pull request Jul 7, 2024
leanprover-community-mathlib4-bot added a commit to leanprover-community/mathlib4 that referenced this pull request Jul 7, 2024
@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit 4ce509b.
There were significant changes against commit 62c5bc5:

  Benchmark                  Metric                  Change
  ====================================================================
- import Lean                branch-misses             2.4%   (11.4 σ)
- import Lean                branches                  1.7%   (15.7 σ)
- import Lean                instructions              1.7%   (15.1 σ)
- lake build clean           instructions              8.1%   (81.0 σ)
- lake build clean           task-clock               54.8%   (39.0 σ)
- lake build clean           wall-clock               15.7%   (17.2 σ)
- language server startup    branch-misses            10.6%   (15.1 σ)
- language server startup    maxrss                   31.9% (1184.3 σ)
- language server startup    task-clock               34.6%   (10.5 σ)
- reduceMatch                instructions              3.5%   (41.7 σ)
- reduceMatch                maxrss                   20.7%   (12.8 σ)
- stdlib                     attribute application    17.0%   (20.8 σ)
- stdlib                     dsimp                    18.1%   (37.8 σ)
- stdlib                     instructions              5.5%  (232.5 σ)
- stdlib                     maxrss                   28.3%   (33.6 σ)
- stdlib                     tactic execution         23.9%  (194.3 σ)
- stdlib                     task-clock               31.4%   (49.1 σ)
- stdlib                     type checking            18.4%   (55.6 σ)
- stdlib                     wall-clock               14.3%   (32.8 σ)
- tests/bench/ interpreted   instructions              1.9%  (181.9 σ)
- tests/bench/ interpreted   maxrss                   12.3%   (99.1 σ)

@Kha Kha force-pushed the parallel-linting branch 2 times, most recently from b98f956 to 656ff8f Compare November 27, 2024 16:06
@Kha
Copy link
Member Author

Kha commented Nov 27, 2024

!bench

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit 656ff8f.
There were significant changes against commit 88e3a2b:

  Benchmark                  Metric                  Change
  =====================================================================
- big_do                     branches                  2.0%   (231.0 σ)
- big_do                     instructions              1.8%   (200.9 σ)
- big_do                     maxrss                    2.7%   (345.6 σ)
- big_do                     task-clock               10.0%    (42.7 σ)
- big_do                     wall-clock                9.4%    (38.2 σ)
+ big_omega.lean             branch-misses            -1.8%   (-39.5 σ)
- big_omega.lean             branches                  4.3%   (295.2 σ)
- big_omega.lean             instructions              4.3%   (282.9 σ)
- big_omega.lean             maxrss                    4.4%  (1723.6 σ)
+ bv_decide_realworld        task-clock               -3.0%   (-17.2 σ)
+ bv_decide_realworld        wall-clock               -3.0%   (-19.9 σ)
- lake config elab           maxrss                    1.7%  (1682.7 σ)
- language server startup    branch-misses             2.2%    (38.5 σ)
- language server startup    maxrss                    1.6%    (70.4 σ)
- reduceMatch                maxrss                    1.7%   (342.7 σ)
- stdlib                     attribute application     3.5%    (12.5 σ)
+ stdlib                     instantiate metavars    -10.2%   (-33.7 σ)
- stdlib                     instructions              1.7%  (3183.7 σ)
- stdlib                     maxrss                   28.3% (17764.5 σ)
- stdlib                     task-clock                5.2%    (39.3 σ)
- stdlib                     wall-clock                2.5%    (12.0 σ)
- tests/bench/ interpreted   instructions              1.9%   (399.2 σ)
- tests/bench/ interpreted   wall-clock               10.7%    (27.4 σ)

@leanprover-community-bot
Copy link
Collaborator

leanprover-community-bot commented Nov 27, 2024

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 88e3a2b1ab287020d5a393dc53c9441aa868681f --onto 9a17919ef11c2dba824498229633b8333a0b53d9. (2024-11-27 16:32:37)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 88e3a2b1ab287020d5a393dc53c9441aa868681f --onto 609346f5e03f3f2d559e7cc905f72e950c3ed8c4. (2024-11-28 10:36:30)
  • 💥 Mathlib branch lean-pr-testing-4460 build failed against this PR. (2024-11-28 14:24:10) View Log
  • ✅ Mathlib branch lean-pr-testing-4460 has successfully built against this PR. (2024-11-28 15:15:21) View Log
  • 💥 Mathlib branch lean-pr-testing-4460 build failed against this PR. (2024-11-29 13:58:33) View Log
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase 86f303774a7fbf38406ae13bd4b66f2753643a45 --onto 6d495586a1836da3e12efb4fbf9946bd9088ac5f. (2024-11-29 16:18:28)
  • ✅ Mathlib branch lean-pr-testing-4460 has successfully built against this PR. (2024-11-30 12:40:50) View Log

@Kha
Copy link
Member Author

Kha commented Nov 27, 2024

!bench

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit 1890ae5.
There were significant changes against commit 88e3a2b:

  Benchmark                  Metric                  Change
  ====================================================================
- big_omega.lean             branch-misses             6.0%   (51.4 σ)
- big_omega.lean             branches                  2.9%  (146.9 σ)
- big_omega.lean             instructions              3.1%  (214.8 σ)
- big_omega.lean             maxrss                    4.5% (1090.8 σ)
- big_omega.lean             task-clock                6.7%   (11.0 σ)
- big_omega.lean             wall-clock                6.7%   (11.0 σ)
+ big_omega.lean MT          branches                 -1.1%  (-48.5 σ)
- reduceMatch                maxrss                    1.8%  (886.2 σ)
- stdlib                     attribute application     3.7%   (20.0 σ)
- stdlib                     instructions              1.4%  (211.7 σ)
- stdlib                     maxrss                   28.4% (5084.2 σ)
- stdlib                     task-clock                4.5%   (34.5 σ)
- tests/bench/ interpreted   instructions              1.9%  (285.5 σ)

@Kha Kha force-pushed the parallel-linting branch 3 times, most recently from 81b9a0c to 4029fa6 Compare November 28, 2024 13:39
leanprover-community-mathlib4-bot added a commit to leanprover-community/batteries that referenced this pull request Nov 28, 2024
@Kha Kha force-pushed the parallel-linting branch from 4029fa6 to aabaa4a Compare November 28, 2024 14:02
@Kha Kha marked this pull request as ready for review November 28, 2024 14:12
@Kha Kha requested a review from digama0 as a code owner November 28, 2024 14:12
@leanprover-community-bot leanprover-community-bot added builds-mathlib CI has verified that Mathlib builds against this PR and removed breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan labels Nov 28, 2024
leanprover-community-mathlib4-bot added a commit to leanprover-community/batteries that referenced this pull request Nov 29, 2024
leanprover-community-mathlib4-bot added a commit to leanprover-community/mathlib4 that referenced this pull request Nov 29, 2024
@leanprover-community-bot leanprover-community-bot added breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan and removed builds-mathlib CI has verified that Mathlib builds against this PR labels Nov 29, 2024
@Kha Kha force-pushed the parallel-linting branch from ddf27d1 to 7fc4163 Compare November 29, 2024 15:57
@Kha
Copy link
Member Author

Kha commented Nov 29, 2024

!bench

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit 7fc4163.
The entire run failed.
Found no significant differences.

@Kha Kha force-pushed the parallel-linting branch from 7fc4163 to f5c78c7 Compare November 30, 2024 11:33
@Kha
Copy link
Member Author

Kha commented Nov 30, 2024

!bench

leanprover-community-mathlib4-bot added a commit to leanprover-community/batteries that referenced this pull request Nov 30, 2024
@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit e369c3e.
There were significant changes against commit 86f3037:

  Benchmark           Metric                    Change
  ================================================================
+ big_do              branches                   -1.4%   (-92.7 σ)
+ big_do              instructions               -1.3%   (-77.9 σ)
- big_do              maxrss                      2.7%   (167.6 σ)
- big_omega.lean      branch-misses              10.8%    (12.8 σ)
- big_omega.lean      branches                    1.3%    (37.4 σ)
- big_omega.lean      instructions                1.6%    (62.5 σ)
- big_omega.lean      maxrss                      4.5%  (6452.9 σ)
+ big_omega.lean MT   branches                   -1.2%   (-72.0 σ)
+ big_omega.lean MT   instructions               -1.1%   (-45.9 σ)
- lake build clean    task-clock                  7.0%    (13.0 σ)
- lake config elab    maxrss                      1.8%   (475.0 σ)
- reduceMatch         maxrss                      1.7%   (418.8 σ)
- stdlib              fix level params            1.9%    (11.9 σ)
- stdlib              maxrss                     28.3% (71098.9 σ)
- stdlib              process pre-definitions     2.4%    (13.0 σ)
- stdlib              task-clock                  3.3%    (23.8 σ)

leanprover-community-mathlib4-bot added a commit to leanprover-community/mathlib4 that referenced this pull request Nov 30, 2024
@leanprover-community-bot leanprover-community-bot added builds-mathlib CI has verified that Mathlib builds against this PR and removed breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan labels Nov 30, 2024
@Kha Kha added this pull request to the merge queue Dec 2, 2024
Merged via the queue into leanprover:master with commit 0b8f50f Dec 2, 2024
19 checks passed
@Kha Kha deleted the parallel-linting branch December 2, 2024 15:02
github-merge-queue bot pushed a commit that referenced this pull request Dec 11, 2024
Fixes #4460 (and similar future changes) making prior messages
inaccessible to metaprograms such as linters
luisacicolini pushed a commit to opencompl/lean4 that referenced this pull request Jan 21, 2025
Fixes leanprover#4460 (and similar future changes) making prior messages
inaccessible to metaprograms such as linters
JovanGerb pushed a commit to JovanGerb/lean4 that referenced this pull request Jan 21, 2025
This PR runs all linters for a single command (together) on a separate
thread from further elaboration, making a first step towards
parallelizing the elaborator.
JovanGerb pushed a commit to JovanGerb/lean4 that referenced this pull request Jan 21, 2025
Fixes leanprover#4460 (and similar future changes) making prior messages
inaccessible to metaprograms such as linters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
builds-mathlib CI has verified that Mathlib builds against this PR changelog-language Language features, tactics, and metaprograms toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants